Emacs skeleton for hugo, updated

A followup to this post on emacs and hugo. One thing that bugged me about that script was that I couldn’t get the timestamp right. It turns out the reason is described by Xah in this post at ergoemacs.

Note, for the time zone offset, both the formats 「hhmm」 and 「hh:mm」 are valid ISO 8601. However, Atom Webfeed spec seems to require 「hh:mm」.

Here’s the correct timestamp function from Xah:

(defun current-date-time-string ()
  "Returns current date-time string in full ISO 8601 format.
Example: 「2012-04-05T21:08:24-07:00」.

Note, for the time zone offset, both the formats 「hhmm」 and 「hh:mm」 are valid ISO 8601. However, Atom Webfeed spec seems to require 「hh:mm」."
  (concat
   (format-time-string "%Y-%m-%dT%T")
   ((lambda (ξx) (format "%s:%s" (substring ξx 0 3) (substring ξx 3 5))) (format-time-string "%z")) )
   )

And the corrected skeleton:

(define-skeleton kjs/yaml-skeleton
  ""
  ""
  "---
title: \"\"
description: \"\"
date: \""
(current-date-time-string)
"\"
tags: []
---

")

Programming: More Emacs for Hugo Scripting

Converting Youtube WordPress Links to iframe Embeds

Yesterday’s DIY involved converting posts from dreamwidth to hugo (actually jekyl). Most of the heavy lifting was done by Thomas Frössm’s exitwp. Once everything was exported into jekyll-flavored markdown, one minor gap was youtube embeds. First, a skeleton for wrapping the iframe around a youtube movie code.

Code on pastebin to get around wordpress sanitation

Then, a function to do it automatically.

(defun kjs/youtube-convert ()
  (interactive)
  (re-search-forward "^http.://.*youtube.*v=")
  (let (p1 p2 myLine code)
    (setq p1 (line-beginning-position) )
    (setq p2 (line-end-position) )
    (setq myLine (buffer-substring-no-properties p1 p2))
    (setq code (replace-regexp-in-string ".*v=" "" myLine))
    (forward-line 2)
    (kjs/youtube-embed)
    (insert code)
    (beginning-of-line)))

Back/Next Page Navigation

How to do back/next pagination came from a snippit by Justin Dunham.

Create a Tag List on a Post

How to create a tag list on a post. Here is the template code in /layouts/_defaults/single.html:

More pastebin

And the css:

.tags ul {
    padding-left: .5rem;
}

.tags {
    display: inline;
    font-size: 70%;
    background-color: #ddd;
}

.tags li {
    display: inline;
}

Archive Page by Date

Figuring out exactly how to get an archive page working took a lot of frustrating searches. I finally found the right template at: /layouts/_defaults/list.html:

More pastebin (see second listing)

Downloading Images from WordPress

exitwp doesn’t grab images. So use grep to get all of the image urls and curl to download them into the images folder.

egrep -oh "https://[^)]+files.wordpress.com/[^)]+.jpg" ~/Dropbox/hugo-blog-test/content/post/*.markdown | xargs -n 1 curl -O

Bulk Edit Files to Point to the Image Folder

Run in the posts folder.

perl -p -i.bak -e 's/https:\/\/[^)]+files.wordpress.com\/[^)]+\//\/images\//' *.markdown 

Emacs Functions for Blogging with Hugo

I’ve been thinking a bit about moving my blog yet again. This time to hugo for a variety of reasons:

  1. I don’t use more than 10% of the features offered by WordPress.
  2. I have very mixed feelings about blog-hosted discussions these days.
  3. WordPress seems to be an easy target for hacks these days.

Most of my personal writing gets composed in emacs as markdown first anyway, so the first set of functions are intended to ease conversion by generating yaml front matter (title and date) for a post from the filename and the first heading. My filenames tend to start with a date stamp YYYYMMDD for ease of searching.

;;functions for compiling blog posts with hugo. 
;;convert dates from YYYYMMDD to YYYY-MM-DD
(defun kjs/date-convert (str)
  (let ((year (substring str 0 4))
    (month (substring str 4 6))
    (day (substring str 6 8)))
    (concat year "-" month "-" day)))

;;get the title from the first header.
(defun kjs/get-title ()
  ;;(interactive)
  (save-excursion
    (re-search-forward "^#")
    (substring (thing-at-point 'line) 2)))

;;add yaml front matter. 
(defun kjs/yaml-add ()
  (interactive)
  (goto-char (point-min))
  (let ((date (kjs/date-convert (buffer-name)))
    (title (kjs/get-title)))
    (insert (concat "---
title: \"" title "\"
description: \"" title "\"
date: \"" date "\"
---" )))
  (re-search-forward "^#")
  (move-beginning-of-line nil)
  (kill-line))

Next, a skeleton and abbreviation for creating new posts:

(defvar current-date-time-format "%F"
  "Format of date to insert with `insert-current-date-time' func
See help of `format-time-string' for possible replacements")

(define-skeleton kjs/yaml-skeleton
  ""
  ""
  "---
title: \"\"
description: \"\"
date: \""
(format-time-string current-date-time-format (current-time))
"\"
tags: []
---

")

(use-package markdown-mode 
  :config
  (define-abbrev markdown-mode-abbrev-table 
    "yamhead" "" 'kjs/yaml-skeleton))

And, a function for using ido completion to insert images:

(defun kjs/insert-image ()
  ""
  (interactive)
  (let ((f (ido-read-file-name "Filename: "
               "~/Dropbox/hugo-blog-test/static/images/")))
    (insert (concat
         "![]("
         (replace-regexp-in-string ".*images" "/images" f)
         ")"))))
 

Passphrase Generation Revisited

One of my hobbies is to try out scripts in multiple programming languages, so I found myself dusting off the passphrase generation script. Mostly it’s an exercise in figuring out how languages do things differently. C++ loves iterators. Racket loves lists. Common-lisp makes some things easy and some things really hard. Python loves list comprehension. I’m still figuring out go. So I write the script, and then try to tweak it to find out which parts of the language are best optimized. Some lessons on this round.

The problem

  1. read a literary text file
  2. create a list of unique words
  3. randomly select 7 words from the list
  4. count the number of unique words.

Lesson 1: If you just want it done, language doesn’t matter.

Timing differences among various implementations ranged from 0.35 seconds (python) to 4 seconds (c++ no optimization). Not that big of a deal except as an object lesson in techniques.

Lesson 2: Python is very good.

My python implementation consistently was the fastest of the bunch. Some of that is due to use of highly optimized c-based functions and list comprehensions. Here is the passphrase creation script (Pastebin.com).

Lesson 3: Imperative design is sometimes faster in racket.

Declaring a variable and updating it with an anti-functional for loop is sometimes faster for this sort of thing, but not by a lot. Racket came in the middle of the pack.


(define (gen-wordlist)
  (define results (mutable-set))
  (for*
      ([line lines]
       [word (regexp-match* word-re line)])
    (let ((lcword (string-downcase word)))
      (when (not (set-member? results lcword))
        (set-add! results lcword))))
  results)
  

Lesson 4: C++ isn’t necessarily faster.

This one surprised me a bit. My C++ implementation read from stdin, and it was much slower than I expected (5 seconds). Using optimization flags -O2 brought it down to python speed on linux. Likely I could get that down faster, but I’m a novice at C++.

Lesson 5: Go concurrency isn’t as easy as it claims to be, and isn’t necessarily faster.

A single-threaded go script performed on par with racket. Trying to multi-thread it took half the night and ended up with a slower program. Again, I’m a novice at go.

Lesson 6: Using common lisp outside of the REPL is harder than it needs to be.

About half of the job was figuring out how to get the script to run outside of SLIME. Problem 1 was ensuring that the cl-ppcre regular expression library was loaded (sbcl used for the following):


;; load quicklisp and cl-ppcre
(eval-when (:compile-toplevel :load-toplevel :execute)
  (load "~/quicklisp/setup.lisp")
  (ql:quickload :cl-ppcre))

Problem 2, properly handle relative filenames from the command line:


(merge-pathnames ; expand relative paths. 
    (car (last sb-ext:*posix-argv*))
    *default-pathname-defaults*))

Problem 3, make it executable by saveing an executable core image

Problem 4, exit on error rather than launch an interactive debugger.


(defun lines (filename)
  "Get the contents of filename as a list of lines."
  (handler-case ;catch errors and quit if the filename is wrong.
      (with-open-file (in filename)
	(loop for line = (read-line in nil)
	   while line collect line))
    (error (err)
      (format t "~a~%" err)
      (sb-ext:exit))))

Problem 5, the sbcl RNG needs to be manually seeded with random data.

(setf *random-state* (seed-random-state t))

Conclusions (such as they are)

Generally speaking, I tend to lean on Racket and branch out into other languages from there. I think python generally is faster and more broadly available.

Largest Remainder

Largest Possible Remainder problem in racket:

;;lr = largest remainder so far lrd = divisor with largest remainder
(define (largest-remainder n d (lr 0) (lrd 0)) 
  (cond
    [(= d 0) (values lr lrd)] ;;catch cases like n = 6, d = 3
    [(> lr d) (values lr lrd)] ;;stop when d is < largest remainder
    [(> (modulo n d) lr)
     (largest-remainder n (sub1 d) (modulo n d) d)] ;;save new largest remainder
    [else 
     (largest-remainder n (sub1 d) lr lrd)])) ;;iterate with d-1

Common Lisp translation:

(defun largest-remainder (n d &optional (lr 0) (lrd 0))
  (cond
    ((<= d 0) (values lr lrd))
    ((< d lr) (values lr lrd))
    (t (let ((r (rem n d)))
	 (if (> r lr)
	     (largest-remainder n (1- d) r d)
	     (largest-remainder n (1- d) lr lrd))))))

Iterative in python.


def largest_remainder(n,d):
    lr = 0
    lrd = 0
    while (d >= lr) and (d > 0):
        if (n % d > lr):
            lr = n % d
            lrd = d
        d -= 1
    return (lr,lrd)

print largest_remainder(20,10)
print largest_remainder(6,3)

When you think the Ashley Madison thing couldn’t get worse…

It gets worse.

Initial reports of the Ashley Madison hack suggested they did one thing right. If I’m reading this report from Ars Technica correctly, they managed to screw even that up by having a second password table with MD5-hashed passwords.

Why is that bad? cryptographic hash functions create unique “signatures” from electronic data. They come in two different varieties. “Fast” algorithms are used to verify the authenticity of gigabytes of data. They’re used to check the integrity of almost everything sent over the internet. They’re designed to be run millions of times a second with minimal memory.

Standard practice for storing passwords is to store a hash “signature” instead of the raw (plaintext) password. You log into a site, it runs the hash function, and compares the signature with the signature stored in its database.

While fast hash algorithms like MD5 are great for checking things like Windows 10 or streaming video. They’re bad for storing passwords. The state of the art in breaking passwords involves making millions of guesses. With MD5 and a graphics card, a password cracker can try over a billion guesses a second.

“Slow” hash functions such as bcrypt or PBKDF2 are designed to take an arbitrary length of time. Instead of a billion guesses per second, a cracker is limited to a few hundred.

While Ashley Madison used bcrypt for their primary password table, they had a second password table with millions of passwords in MD5. As of yesterday, 11.7 million passwords have been hacked.

Not that a matters that much. Users chose really bad passwords. The top 10:

  • 123456
  • 12345
  • password
  • DEFAULT
  • 123456789
  • querty
  • 12345678
  • abc123
  • pussy
  • 1234567

The rest of the top 100 seems to combine numbers, sex, and wishful thinking. Sometimes in one password.