Bash - Scripting notes

author: Paul Kim

categories: bash, linux, mac

tags: bash, linux, mac

bash script to migrate markdown files from old jekyll site to new gatsbyjs site

i made a bash script to reorganize my markdown files in preparation for migrating my site to gatsbyjs. i learned a lot of things and should probably blog about it when I have the time.

so what does this script do?

I had all my markdown files under a folder called _posts. Some markdown files are "posts" and others are "notes". The markdown files that are "notes" ended in -notes.md (i.e., 2019-01-01-android-notes.md)

Here is what this script does:

add type: note front-matter to all markdown files ending in -notes.md and add type: post front-matter to the remaining markdown files.
add 'date: YYYY-MM-DD' to front-matter based on the date format of the markdown file.
create a folder with the same name as the markdown file. then move the markdown file under that folder and rename as index.md.
for all folders that end in -notes, create a new folder called notes and put all those folders ending in -notes under the folder called notes.
for all folders that are not under notes, move them to a new folder called blog.

#!/usr/bin/env bash

target="/home/paul/Desktop/_posts"

# add 'type: post' or 'type: note' to front-matter
for f in "$target"/*; do
  if [ -f "$f" ] && [ ${f: -9} == "-notes.md" ]; then
    sed -i '0,/---/a type: note' "$f"
  else
    sed -i '0,/---/a type: post' "$f"
  fi
done

# add 'date: YYYY-MM-DD' to front-matter
for f in "$target"/*; do
  if [ -f "$f" ] && [ ${f: -3} == ".md" ]; then
    # echo "$f"
    # echo $(dirname $f)
    # echo $(basename $f)
    ff=$(basename $f)
    date=${ff:0:10}
    # echo $date
    # sed -i '/^author:.*/a date: '"$date"'' "$f"
    sed -i '0,/^\(author:.*\)/s//\1\ndate: '"$date"'/' "$f"
  fi
done

# # remove lines containing {::options auto_ids=false}
# # skip this step and just do it in VS Code after import
# # because it is so much easier. and we have to do it for
# # many other lines besides these ones.
# pattern="{::options auto_ids="
# # let count=0
# for f in "$target"/*; do
#     if [ -f "$f" ] && [ ${f: -3} == ".md" ]; then
#         if grep -q "$pattern" "$f"; then
#             # echo $(basename $f)
#             e=$(grep --max-count=1 --line-number "$pattern" "$f" | cut -d: -f1)
#             # echo $e
#             s=$((e-1))
#             # echo $s
#             sed -i ''"$s"','"$e"'d' "$f"
#             # let count=count+1
#         fi
#     fi
# done
# # echo "Count: $count"

for f in "$target"/*; do
  if [ -f "$f" ] && [ ${f: -3} == ".md" ]; then
    # echo "$f"
    # echo $(dirname $f)
    # echo $(basename $f)
    ff=$(basename $f)
    fff="${ff#[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-}"
    # echo "$fff"
    ffff="${fff%.*}"
    # echo $(dirname $f)/"$ffff"
    mkdir $(dirname $f)/"$ffff"
    mv "$f" $(dirname $f)/"${ffff}/index.md"
  fi
done

for d in "$target"/*; do
  if [ -d "$d" ] && [ ${d: -6} == "-notes" ]; then
    # echo "$d"
    # echo $(dirname $d)
    # echo $(basename $d)
    mkdir -p $(dirname "$d")/notes
    mv "$d" $(dirname "$d")/notes/$(basename "$d")
  fi
done

for d in "$target"/*; do
  if [ -d "$d" ] && [ "$d" != $(dirname "$d")/notes ]; then
    # echo "$d"
    # echo $(dirname $d)
    # echo $(basename $d)
    mkdir -p $(dirname "$d")/blog
    mv "$d" $(dirname "$d")/blog/$(basename "$d")
  fi
done

so what does this script do?

The following script removes any line containing {::options auto_id=" and the line before it from all markdown files.

loop through all files and directories
if it is a file that ends in .md and contains the pattern {::options auto_id=", get the line number
use the line number to delete the line and the line before it (2 lines total)

#!/usr/bin/env bash

PATTERN="{::options auto_ids="

# loop through all .md files.
# if file contains pattern, get line number
# use line number to delete lines in file with sed
for f in *; do
  if [ -f "$f" ] && [ ${f: -3} == ".md" ]; then
    if grep -q "$PATTERN" "$f"; then
      # echo "$f"
      e=$(grep --line-number "$PATTERN" "$f" | cut -c1)
      # echo $e

      s=$((e-1))
      # echo $s
      sed -i ''"$s"','"$e"'d' "$f"
      # sed -i '/^{::options auto_ids=/d' "$f"
    fi
  fi
done

# for f in *; do
#   if [ -f "$f" ] && [ ${f: -3} == ".md" ]; then
#     ff="${f#[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]-}"
#     fff="${ff%.*}"
#     mkdir "$fff"
#     mv "$f" "${fff}/index.md"
#   fi
# done

# for d in *; do
#   if [ -d "$d" ] && [ ${d: -6} == "-notes" ]; then
#     mkdir -p _notes
#     mv "$d" "_notes/$d"
#   fi
# done

# for d in *; do
#   if [ -d "$d" ] && [ "$d" != "_notes" ]; then
#     mkdir -p _blog
#     mv "$d" "_blog/$d"
#   fi
# done

Notes