Powershell eccentricitys

• Get-Help (linux of -h or --help or man x)
  • Get-Help ls or Get-Help ls --full
  • -Force on ls is like ls -a
    • example: ls -Force C:\ to see ProgramData
• As a general rule just change forward slash to backslash in Powershell and things just kind of work as you'd expect. This is because while powershell commands are weird, but have a ton of aliases that are mapped to the UNIX equivalents.
• Some exceptions to the rule:
  • -f becomes -Force
  • -r becomes -Recurse
    • Often arguments needs to be at the end of a command, ex: rm -r FILE -Recurse
• Powershell doesn't have less command, just the old bash more, which does still work.
• Select-String - ps version of grep
• -Filter lets you search for files in a directory, ex: ls C:\ -Recurese - Filter *.exe will give you so much data that you won't be able to use it for anything if you don't pipe it through more.

Escape Characters

1. In Linux \ is the escape character.

   mkdir My\ Folder

2. Windows in it's normal weird fashion uses a backtick `

   mkdir My` Folder

• history Ctrl+R allows searching through recent command history in Powershell and bash.
• wildcard * is handy as hell, cool example:

cp *.jpg ~/Dowloads/

# Copies all jpegs from working directory to ~/Downloads

Linux stuff

• /bin - stores apps
• /etc - system configs
• /home - duh
• /proc - currently running process
• /usr - usr intalled files
• /var - temp stuff (sort of)

ls -l broken down

ls -l permissions broken down

The first letter in the string (d in the above example) indicates the type of file that is being listed.

• d = directory
• - = regular file
• l = symbolic link

The next nine characters are broken into three categories, permissions for the:

1. file owner
2. group
3. other users

• r = read
• w = write
• x = execute

- indicates that a permission does not exist, for example: r-x would indicate rights to read and execute, but not to write. In the example above (drwxr-xr-x), we can learn that:

1. The file is a directory.
2. The owner has full rights: read, write, execute
3. The group has read and execution rights, not write rights.
4. All other users also have read and execution rights, not write rights.

Viewing Files

• cat - prints full file
• head - first 10 lines
  • head -n 1 FILE - prints specific number of lines (10 is just a default)
  • head --lines 1 FILE - same thing
• tail - last 10 lines
  • tail -n 1 FILE - prints specific number of lines (10 is just a default)
  • tail --lines 1 FILE - same thing
• less - fills window with text file
  • Bash also has the legacy more command, but why bother if you don't have to?

Grep and His Powerful Friends

Way better than the powershell set-up (or at least less weird). Some examples follow:

• grep "America is doomed" Documents/journal/*.md - finds every-time I wrote a journal entry after watching the news in 2026.
  • grep "America is doomed" Documents/journal/*.md | less - give me an output that I can actually parse easily.
  • grep -c "America is doomed" Documents/journal/*.md - would just brake down how mnay times I've used the phrase in a journal.
    • Example Output:

    Documents/journal/volume-03.md:20
    Documents/journal/volume-04.md:122
    Documents/journal/volume-05.md:36
    Documents/journal/volume-06.md:79
    Documents/journal/volume-07.md:22

  • grep -i - case insensitive
  • grep -n pattern - show line numbers with output
  • grep -w - only returns if pattern matches exactly
    • grep -w data app.log wouldn't return the word "database" as a result
  • Showing lines before or after pattern is found:
    • Before: grep -B 2 "data" app.log - shows two lines before "data"
    • After: grep -A 85 "data" app.log - shows eighty-five lines before "data"
  • grep "America is doomed | sort --unique - filters out anytime I've just timed "America is doomed" and ended the entry there.

Creating a New Branch

1. Navigate to git repo:

cd REPO

2. Create new branch:

git checkout -b NEW-BRANCH

3. Push new branch:

git push origin NEW-BRANCH

(Optionally) Create a Branch from a Specific Commit

git checkout -b NEW-BRANCH HASH

Swap Between Branches

git checkout BRANCH-NAME

Merge Changes to master/main Branch

1. Change back to main branch:

git checkout master

2. Merge it:

git merge branchNAME

Locating a File to Download

1. Toggle Developer Mode in any modern full-featured browser. Developer mode can be triggered in Chromium or Firefox by pressing F12.
2. Swap to the Network tab.
3. Reload web page if necessary and ensure video playback has been triggered.
4. Monitor for any video files, using URL filters andior the Media category in the Developer menu. <<<<<<< HEAD
5. Select individual requests and view the headers until a url with a video file is included, the video file will often be obscured, as in the following example: https://video.com/something.php?file=name.mp4&extra_bs =======
6. Select individual requests and view the headers until a url with a video file is included, the video file will often be obscured, as in the following example: https://site.com/control.php?file=filename.mp4&acctoken=token

development

An example using the network tab in Firefox's Dev Tools to located a segmented video stream when a video starts playing on an example website.

Downloading the Video File

1. Using either curl or wget use the full URL to download the video file.

<<<<<<< HEAD
curl -L -O "https://video.com/something.php?file=title.mp4&token=token"
=======
curl -L -O "https://sie.com/control.php?file=filename.mp4&acctoken=token"
>>>>>>> development

or

<<<<<<< HEAD
wget --content-disposition "https://video.com/something.php?file=title.mp4&token=token" 
=======
wget --content-disposition "https://site.com/control.php?file=filename.mp4&acctoken=token" 
>>>>>>> development

ensure quotes around the video file URL are in place

2. Multiple attempts may be neccasary if their are multiple video files located (ads in front of video, et cetera), however once the curl or wget process has started the file-size is usually a pretty good giveaway (18MB vs 250MB is a pretty clear difference and easy to discriminate against).
3. When the download is complete, often times you will be left with something other than a video file, example: remote_control.php would most likely be the filename if downloading a video with the previously referenced sample URL. The file-size here is once again a pretty clear give-away, as most .php files won't be several hundred megabytes.
4. Change the file-name and extension as you will, ensuring that the .php extension is changed to a video container. I have found that .mp4 or .mov will almost always result in a playable video, but depending on the format of the original file, a few tries might be necessary. In any case, simply matching the file extension of the original file should always work.

Sequenced Videos

Some websites don't list a full video file in the Network tab of the Developer Menu. Rather than hosting a full file, it's only possible to access segmented video clips. This however does not mean that we cannot still download a full video.

1. First find the sequence of video clips, for example the following URL might show up: https://video.net/video.mp4/seg-01-av.ts
2. Create a list of all segmented files, by first downloading an individual segment:

curl -L -O "https://video.net/video.mp4/seg-01-av.ts"

3. Once you have one video segment, you can calculate or roughly guess the video segment in question. For example if the first segment was roughly two seconds, and the video was about thirty minutes long, we can reasonably guess that there would be about nine hundred individual segments.
4. Create a bash script to download all of the segments:

#!/usr/bin/env bash

BASE_URL="https://video.net/video.mp4"

for i in $(seq -w 1 900); do
    curl -L -O "$BASE_URL/seg-$i--av.ts"
done

5. Run the script.
6. Add all the downloaded files into a properly organized text document. This function will organize all the files, regardless of whether or not they are numbered with proceeding zeros:

printf '%s\n' seg-*-av.ts | sort -V | sed "s/^/file '/; s/$/'/" > files.txt

7. Combine all the files into one with ffmpeg:

ffmpeg -f concat -safe 0 -i files.txt -c copy output.mp4

1. Follow the Docker website's documentation, it's solid & also includes instructions to remove previous iterations of Docker on all the other supported distributions.

Install Calibre-web via the Docker Image

• Note: Calibre-web can be installed via a python pip package, however the Docker image is easier to set up and run as a background process on a Linux server.

1. Create a new directory for the calibre-web app to live in:

mkdir Docker/claibre-web
cd Docker/claibre-web

2. Set up a docker-compose.yml file in app directory:

touch docker-compose.yml

3. Ensure to make required changes to the volumes: section of the default docker-compose.yml:

name: calibre-web

---
services:
  calibre-web:
    image: lscr.io/linuxserver/calibre-web:latest
    container_name: calibre-web
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      - DOCKER_MODS=linuxserver/mods:universal-calibre
    volumes:
      - /home/USER/Docker/calibre-web/config:/config
# ~/Docker/calibre-web/ will map the Calibre config directory to /config in the app directory
      - /home/USER/Documents/Books:/books 
# ~/Documents/Books = Pre-existing Calibre libary directory including metadata.db file
    ports:
      - 8083:8083
    restart: unless-stopped

4. Start the Calibre server:

docker compose up -d

Set-up the Calibre Library

1. From inside the server's LAN access the server via: http://serverIP:8083
2. Login with the default credentials:

Username: admin
Password: admin123

3. When prompted set the Calibre Database Location to /books. Link to a pre-existing calibre libary in the docker-compose.yml, in the above example: /home/USER/Documents/Books is mapped to /books.

Databse config setup in the Calibre Web UI

Access the Calibre Server Remotely

The easiest & most secure way to access the calibre-server remotely is via a VPN. Wireguard is very good for this, & Tailscale is an even simpler abstraction that uses Wireguard as a dependency. Full instructions for installing and configuring Tailscale can be found here.

Using the Calibre-web OPDS Server

Maybe the best thing about the calibre-web server is the fact that it spins up an OPDS server alongside the web UI that can be accessed via many Andoid and iOS eReader apps. This server can be accessed in a browser or in an application by navigating to: http://serverIP:8083/opds

The Yomu app on iOS with the Calibre-web OPDS server linked via a Tailscale VPN connection.

OPDS makes it incredibly easy to browse and download a user's full Calibre library of eBooks in any number of OPDS apps across all platforms. Some of the most popular apps are FBReader on iOS, Moon+ Reader for Android & Thorium Reader for Linux, Windows and Apple Silicon Macs. A much larger list of OPDS applications can be found on the Awesome OPDS GitHub page.

My personal calibre library being accessed via the OPDS server in Yomu

1. Note: Docker Engine is not the same thing that would generally be installed via a standard linux package manager. To remove a previously installed iteration of Docker (at least on Ubuntu), use the command below:

sudo apt remove $(dpkg --get-selections docker.io docker-compose docker-compose-v2 docker-doc podman-docker containerd runc | cut -f1)

2. Follow the Docker website's documentation, it's solid & also includes instructions to remove previous iterations of Docker on all the other supported distributions.

Install Immich

1. Use the Immich website's documentation for installing and running with Docker Compose; once again, this documentation is more than good enough.

Configure Immich

1. Visit port 2283 on the machine that Immich was installed on: http://SERVER.IP.ADDRESS:2283.
2. Follow the prompts to set up an admin & some user accounts & use Immich's documentation if stuck.
3. There are a few different ways to remotely access an Immich server, but one of the easiest & most secure is via a VPN - just do that unless there's a really good reason not to.

• pv. Don't use dd. While it will often work, dd is a 50+ year old command, uses archaic syntax and provides little to no feedback on what it is doing.
• An ISO image (usually .iso, but the process will also work for .raw.xz, and some other archived images).

Steps

1. Insert a removable USB drive, and ensure that there is at least one partition large enough for the ISO image.
2. Use lsblk to check the drive name & partitions (very often a removable drive will be something like /dev/sda or /dev/sdb).

lsblk -f ## List block devices

## Look for something like the partition tree below

NAME        FSTYPE FSVER LABEL UUID  
sda                                   
└─sda1                                                                             

3. If the drive has no partition, cfdisk is a user-friendly TUI tool that can create drive partitions.

sudo cfdisk /dev/sda
## Ensure changes are written before exiting the application

4. Write the ISO image to a partition on the drive:

sudo pv arch.iso -o /dev/sda1

Optional Improvements

• --sync or -Y will use fdatasync to synchronize the buffer cache after every write operation. This will sometimes improve the accuracy of pv's progress bar when writing data to a slow disk (like a USB stick).

sudo pv arch.iso --sync -o /dev/sda1

## or

sudo pv arch.iso -Y -o /dev/sda1

• --stats or -v will show the transfer rate in bytes per second.

sudo pv arch.ios --sync --stats -o /dev/sda1

Required Hardware

• A server. The required software is available on virtually every OS and should run on any modern CPU architecture. This documentation provides instructions for a POSIX-style shell.
• A client machine to push to the server, once again, virtually any OS or CPU architecture will work.

Required Software

• Open SSH
• Git
• A VPN software, this guide uses: Tailscale

Required set-up for both devices

On both the server and any machine you intend to push changes from, you will need to set-up a VPN connection and install OpenSSH, as well as an optional SSH key. The process is identical for each system:

Install Tailscale

curl -fsSL https://tailscale.com/install.sh | sh

Authenticate your user account when prompted, or set up a new account on first use.

Install OpenSSH

sudo pacman -S openssh  # Arch Linux

sudo apt update && sudo apt install openssh-server # Debian & Ubuntu

sudo dnf install openssh-server  # Fedora

sudo pkg install openssh  # FreeBSD

brew install openssh  # Homebrew

nix profile install nixpkgs#openssh  # Nix

Server Set-up

Remote into the server using SSH and the server's Tailscale generated IP address. This documentation will use the user name: "user" & a fake IP address of "100.27".

tailscale status  # To locate the server's IP

ssh user@100.27

1. Set up a location in which to store Git repositories
2. Create a directory with the name of the repository
3. Initiate a bare git repository

mkdir Repos && cd Repos
mkdir "REPO-NAME" && cd "REPO-NAME"

git init --bare

Client Set-up

Create an empty directory with the same name (REPO-NAME) anywhere on the client system.

mkdir REPO-NAME && cd REPO-NAME

Once inside of that directory run the following commands:

git init  # Initiates the new repository
git add -A   # Stages all files
git commit -m "Initial commit"  # Commit's those changes

git remote add origin user@100.27:/home/user/Repos/REPO-NAME/

git branch  # Depending on Git version, changes might be commited to the "main" or "master" branch by default. Note which branch this repository defaulted to.

git push -u origin master 
# or
git push -u origin main

After this set-up any new changes can be pushed to the private server with the standard git push command, with no additional flags or options required.

Note: Git will only accept one origin to push to. If a typo or other error is made while running remote add origin, this can be fixed by running:

git remote set-url origin user@server:/home/user/Repo/REPO-NAME/

git remote -v  # Can be used to confirm that a typo was made upon initial set-up as well

Migrating an Existing Repository to Server

• Change the remote origin
• Ensure the primary branch names match up
• Push to server

git remote set-url origin user@server:/home/user/Repo/EXISTING-REPO/
git remote -v  # To check url & confirm before pushing

git branch  # To check primary branch name, almost always "master" or "main" 

# The primary branch names must line up, to change run
git branch -m BRANCH-NAME

git push -u origin BRANCH-NAME

Cloning Repo on a New Client

Ensure SSH and Tailscale are installed and configured on any new machine, and then run:

git clone user@100.27:/home/user/Repos/REPO-NAME/

Authentication with a SSH key

Authenticating transfers and remote operation via SSH keys is not only much more secure than a standard password, but also much better for any sort of CI, CD or cron job automation that a user might want to set up in the future. Luckily the ssh-keygen utility is packaged with OpenSSH upon install. This makes it very easy to create a SSH key on both the server and client.

ssh-keygen -t ed25519 -a 100 -C "you@mail.com"

# If you are using a legacy system that does not support the Ed25519 hash algorithm use rsa
ssh-keygen -t rsa -b 4096 -a 100 -o -C "you@mail.com"

# If you want to use a hardware key for 2FA
ssh-keygen -t ed25519-sk -C "you@mail.com"

After creating the key, add the client's public SSH key to the server's list of authorized keys. This can be done by manually copying and pasting the full public key, but OpenSSH has a built in utility that makes the process much easier.

ssh-copy-id user@100.27

Note: If a repository was initially cloned or initiated using HTTPS, it might be necessary to add the ssh:// prefix to the URL. This is not necessary when using the instructions provided here and pushing to origin via a VPN connection, but still helpful information to have access to.

git remote set-url origin ssh://makc@100.27:/home/user/Repo/REPO-NAME/

Issue

By default demnu (and most other minimal app launchers) look(s) for packages installed in /usr/bin. This is where most packages installed via apt, pacman or other package managers generally write to. However when installing apps with services like Flatpak or Canonical's Snap Store, the appImage is often placed elsewhere.

Note: I have found that Snap packages specifically often do launch via dmenu without issue, though not always. This fix will apply to any application that is installed to a non-standard location.

Steps

Locate your newly installed package:

• Flatpak installs to /var/lib/flatpak/exports/bin
• Snap Store typically installs to either /var/lib/snapd/snap or /snap
• Blackmagic installs Davinci Resolve in /opt/resolve/bin

Create a symlink to /usr/bin, where dmenu can find it:

sudo ln -s /var/lib/flatpak/exports/bin/com.app.NAME /usr/bin/appNAME

Replace appNAME with whatever you want to type into dmenu to launch the app.

Examples

PlexAMP (Flatpak)

sudo ln -s /var/lib/flatpak/exports/bin/com.plexamp.Plexamp /usr/bin/plexAMP

Davinci Resolve (Blackmagic installer)

sudo ln -s /opt/resolve/bin/resolve /usr/bin/resolve

Source

I found the original fix from a gist written by Swati Jaiswal.

Required Software

• Davinci Resolve
• Proprietary Nvidia, or AMD Drivers
• unzip

Important Note: Davinci Resolve requires a GPU with OpenCL or CUDA support. Full compatibility information can be found on the Khronos Group's list, but running Resolve on Linux will likely require a discrete GPU & may require configuring a laptop with hybrid graphics to run using only the GPU for display output.

Important Note: Resolve is only supported running with proprietary Nvidia drivers. Resolve will not run with Nouveau drivers installed.

Important Note: While Resolve included full Wayland support in Resolve 19, many users have reported issues. This documentation covers installing Resolve in an X11 environment.

Installation

1. Ensure that the appropriate display driver's & unzip are installed.

# Arch (Nvidia)
sudo pacman -Syu nvidia nvidia-utils cuda opencl-nvidia unzip

# Arch (AMD)
sudo pacman -Syu rocm-opencl-runtime rocm-hip-runtime unzip

# Fedora (Nvida)
sudo dnf install -y \
https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

sudo dnf install -y akmod-nvidia xorg-x11-drv-nvidia-cuda

# Fedora (AMD)
sudo dnf install -y rocm-opencl rocm-hip
sudo usermod -aG render,video $USER

# Ubuntu (Nvida)
sudo apt install nvidia-driver-550 nvidia-cuda-toolkit

# Ubuntu (AMD)
sudo apt install rocm-opencl-runtime rocm-hip-runtime

2. Download Davinci Resolve Linux from Black Magic Design's website.

Note: The process is identical for the free version of Davinci Resole and Davinci Resolve Studio.

3. Extract the installer & make it executable.

cd ~/Downloads/ # Navigating to Downloads
unzip DaVinci_Resolve_*_Linux.zip # Unzipping zip file
chmod +x DaVinci_Resolve_*_Linux.run # Making install script executable

4. Install.

sudo SKIP_PACKAGE_CHECK=1 ./DaVinci_Resolve_*_Linux.run

Post Install Fixes

Fedora Libraries Fix

If installing on Fedora Linux it may be necessary to replace older libraries that conflict with Fedora workstation:

cd /opt/resolve/libs
sudo mkdir disabled-libraries
sudo mv libglib* libgio* libgmodule* disabled-libraries # Move outdated libraries to disabled-libraries

sudo dnf install -y libxcrypt-compat # Instal updated libraries

Display Scaling Workaround

If using Davinci Resolve on a HiDPI display, scaling issues may be encountered. This is due to the fact that Resolve uses QT's GUI toolkit, but often ignores userspace environment variables to control QT apps. A quick workaround is to create a wrapper script that will change the display scaling when launching Resolve, and revert the scaling when Resolve is closed:

#!/bin/bash
xrandr --output DP-2 --scale 0.5x0.5 # Scale screen down by 2x
/opt/resolve/bin/resolve # Launch Resolve
xrandr --output DP-2 --scale 1x1 # Restore native scale

Launching

When installing Davinci Resolve in a full Desktop environment, Resolve will most likely not launch. The easiest solution is to add an entry in /usr/bin for Resolve.

sudo ln -s /opt/resolve/bin/resolve /usr/bin/resolve

Required Software

• ffmpeg: For transcoding media
• xrandr: For setting a custom display resolution on X11
• wlr-randr: For setting a custom display resolution on Wayland

Optional

• OBS: For recording screen
• Davinci Resolve: An all around awesome NLE that runs on Linux, proprietary
• Kdenlive: A less than ideal, but usable NLE that is open-source

Installation

Note: Installing Davinci Resolve on Linux requires a fairly specific process, documentation for installing can be found here.

Note: cvt is used in this documentation and installed with the X11 utilities

# Arch Linux
sudo pacman -Syu
sudo pacman -S obs-studio kdenlive ffmpeg xorg-xrandr

# Fedora 
sudo dnf upgrade --refresh -y
sudo dnf install -y \
https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm \
https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm 

sudo dnf install -y obs-studio kdenlive ffmpeg xorg-x11-server-utils


# Ubuntu
sudo add-apt-repository universe
sudo apt update && sudo apt upgrade -y
sudo apt install -y obs-studio kdenlive ffmpeg x11-xserver-utils

Changing A Display's Native Resolution

Depending on the display, it is sometimes possible to use the cvt command to generate a modeline, and then use xrandr's --addmode flag to create a fully custom display resolution. Basic steps can be found below, or more detailed instructions can be viewed here:

cvt 3840 1920 60 # Generating Modeline at 3840x1920 & 60hz
# 3840x1920 59.97 Hz (CVT) hsync: 119.27 kHz; pclk: 629.75 MHz
Modeline "3840x1920_60.00"  629.75  3840 4144 4560 5280  1920 1923 1933 1989 -hsync +vsync

xrandr --output DP-2 --addmode "3840x1920_60.00"  \ # Adding output to X11 display
629.75  3840 4144 4560 5280  1920 1923 1933 1989 -hsync +vsync 

However, if using a display port connection, or a modern laptop that uses an eDP display, it is very often not possible to set a fully custom resolution. However, X11's tools for display scaling can be exploited to achieve virtually the same thing. In the case of the Thinkpad P1 that was used for this documentation, the native resolution is 3840x2160. The desired output video resolution is at a 2:1 aspect ratio, 3840x1920. Dividing the desired vertical resolution by the original vertical resolution, 1920 ÷ 2160, gives a result of 0.888888888889, which can be used as the vertical scale factor in xrandr.

xrandr --output DP-2 --mode 3840x2160 --scale 1x0.8889

Notes:

• The wlr-randr command functions as a clone of xrandr for Wayland, and will accomplish the same goal in Wayland window managers or desktop environments.
• The scale factor will have to be reset after any screen recording.

xrandr --output DP-2 --mode 3840x2160 --scale 1x1

Transcoding H.264

Unless using a custom ffmpeg output, OBS will only output video using the H.264 codec. This is a problem for a number of reasons when building a video creation pipeline on Linux:

1. Fedora, Ubuntu & many other Linux distributions do not ship an iteration of ffmpeg that supports H.264 libraries.
2. Kdenlive will not allow users to edit H.264 video without a transcode.
3. The free edition of Davinci Resolve will not work with H.264 at all when running on Linux.

Much of this is due to a licensing fee structure implemented by Via-LA-- but the end result is that, the first step in any editing pipeline has to be a transcode to escape the limitations of the H.264 codec. NLE friendly alternative codecs include, but are not limited to, AVID's DNxHR, GoPro's CineForm and Apple's ProRes. All three have varying degrees of support across Linux distributions, but ProRes typically works out of the box and is what this documentation will recommend.

ffmpeg -i INPUT.MOV \ # Calls ffmpeg and selects file to transcode 
-c:v prores_ks \ # "c:v" = codec for video, "prores_ks" = ProRes Codec
-profile:v 0 \ # How ProRes sets quality 0=proxy, 1=low, 2=high, 3=lossless
-c:a pcm_s16le \ # "c:a" = codec for audio, "pcm_s16le" is selecting 16-bit audio
OUTPUT.md # Gives a name for output

ffmpeg -i INPUT.MOV -c:v prores_ks -profile:v 0 -c:a pcm_s16le OUTPUT.MOV # Sample ProRes
ffmpeg -i INPUT.MOV -c:v dnxhr -profile:v dnxhr_lb -c:a pcm_s16le OUTPUT.MOV # Sample DNXHQ
ffmpeg -i INPUT.MOV -c:v cfhd -quality medium -c:a pcm_s16le OUTPUT.MOV # Sample Cineform 

Depending on use case, it is sometimes worth creating a bash script to handle this process. If this is desired, the use of input variables will make a script run more efficiently:

ffmpeg -i $1 -c:v prores_ks -profile:v 0 -c:a pcm_s16le "prefix-$(basename "%1")"

# Usage
./scriptname.sh FILENAME.mov # Outputs a transcoded file named, prefix-FILENAME.mov

Bonus: Quick Audio Cleanup in Resolve

1. Drag audio onto timeline, select Fairlight panel.
2. Right-click audio track, select Clip Options, then Normalize Audio.
3. When the pop-up window appears, normalize audio to -1db.
4. Navigate to the Mixer Panel, select the Plus sign to add and effect, EQ menu, select Fairlight EQ.
5. To set up a basic noise gate: mute or reduce audio output on Band 1 & Band 2, which control 100hz & 120hz wavelengths respectively.
6. If there is any hissing or popping in narration, slightly lower the decibel level of Band 6.

Transcoding for Delivery

When exporting from Davinci Resolve, once again we run into issues, as Resolve does not support exporting to the H.264 codec. Luckily, Google's VP9 codecs has nearly all the same compression advantages as H.264 and none of the licensing issues. Unfortunately, Resolve also can't export to the VP9 codec. The easiest workaround is to export ProRes from Resolve and transcode to VP9 using ffmpeg for final delivery to a video sharing site:

ffmpeg -i INPUT.mov -c:v libvpx-vp9 -crf 35 -b:v 0 -c:a libopus -b:a 512k OUTPUT.webm

# The syntax for using an input variable is slightly different here, due to the changing file extension
ffmpeg -i "$1" -c:v libvpx-vp9 -crf 35 -b:v 0 -c:a libopus -b:a 512k "prefix-$(basename "$1" .mov).webm"

Background

Last year I began writing my first novel. My preferred method of writing is to use the Markdown format, and Neovim. This setup is great for writing efficiently and accessing my work on any device. However, I have run into two issues:

1. I prefer to edit with pen & paper. Markdown is not a printer friendly format.
2. If I ever do try to publish my work, no publisher in the world will accept a .md file as a manuscript.

Both of these problems can be solved by periodically converting the Markdown files into an Office format. I prefer the Open Office format (.odt), but this process is nearly identical with Microsoft Word (.docx) or Google Doc (.gdoc) formats.

Required Software

• Pandoc: A tool that will convert Markdown files to Open Office files.
• Libre Office: My office software of choice. Open Office will work just as well.

Optional

• My Open Office Template
• My .lua filter, available as part of my dotfiles.

Installation

# Arch
sudo pacman -Syu; sudo pacman -S libreoffice-still pandoc

# Fedora
sudo dnf upgrade --refresh -y && sudo dnf install -y libreoffice pandoc

# Ubuntu 
sudo apt update && sudo apt upgrade -y && sudo apt install -y libreoffice pandoc

# FreeBSD
su && pkg update && pkg upgrade -y && pkg install -y libreoffice pandoc

# MacOS (via Homebrew)
brew update && brew upgrade && brew install pandoc libreoffice

Procedure

Open Office Template

The first step is to create an Open Office template, or, what Pandoc will refer to as a: reference-doc. To do this, run an existing Markdown project through Pandoc to generate an Open Office file with all of the Markdown elements.

pandoc INPUT.md -o OUTPUT.odt 

Open the exported .odt file in Libre Office and edit each paragraph style that is utilized. This can be done by opening the paragraph style menu, and using the drop down menu to edit each style individually. I have made the template I use available for download.

Store this reference.odt file in an easy to access location, I put a copy of the template in the root directory of any writing project I'm embark on.

Libre Office's style's list

.lua Filter

A problem I encountered early on involved my use of Markdown’s horizontal rule feature. When exporting to an Office format, Pandoc, quite reasonably, renders the horizontal rule as a visible line. However, in my document this element is intended to mark a scene break, which follows specific formatting conventions in a manuscript.

A before and after of the horizontal rule conversion via the lua filter

To avoid making changes to my Markdown documents, I created a .lua filter that can be run with Pandoc, and will convert the horizonal rule (---) in Markdown, to a # symbol, when exporting. In order for this filter to work properly, it is necessary to create a SceneBreak style in the reference.odt template.

-- rule-to-scene-break.lua

function HorizontalRule()
    local para = pandoc.Para({pandoc.Str("#")})
    return pandoc.Div({para}, pandoc.Attr("", {}, {["custom-style"]="SceneBreak"}))
end

Creating the Final Manuscript

Using the Open Office template and .lua filter, convert the Markdown file to an Open Office file. In order to do this, use the standard Pandoc syntax, but provide a link to the reference.odt template, with the --reference-doc= flag, and the .lua filter, using the --lua-filter= flag.

pandoc INPUT.md -o OUTPUT.odt \
--reference-doc=reference.odt \
--lua-filter=/home/makc/.scripts/rule-to-scene-break.lua

Converting to PDF

To create a PDF from the command line, we can use Libre Office in a headless manner. After creating the Open Office Document with Pandoc, run:

libreoffice --headless --convert-to pdf FILENAME.odt

Or, simply create one bash script to handle everything in one motion.

#!/usr/bin/env bash
set -euo pipefail

# Resources
SCRIPTS_DIR="$HOME/.scripts"
REFERENCE_DOC="$SCRIPTS_DIR/reference.odt"
LUA_FILTER="$SCRIPTS_DIR/rule-to-scene-break.lua"

# Arg check
if [[ $# -ne 1 ]]; then
    echo "Usage: $(basename "$0") FILE.md"
    exit 1
fi

INPUT="$1"

if [[ ! -f "$INPUT" ]]; then
    echo "Error: File not found: $INPUT"
    exit 1
fi

# Paths
INPUT_ABS="$(realpath "$INPUT")"
BASENAME="$(basename "$INPUT_ABS" .md)"
WORKDIR="$(dirname "$INPUT_ABS")"

ODT="$WORKDIR/$BASENAME.odt"
PDF="$WORKDIR/$BASENAME.pdf"

# Converting and print to terminal
echo "→ Converting Markdown to ODT..."
pandoc "$INPUT_ABS" \
    -o "$ODT" \
    --reference-doc="$REFERENCE_DOC" \
    --lua-filter="$LUA_FILTER"

echo "→ Converting ODT to PDF..."
libreoffice \
    --headless \
    --convert-to pdf \
    --outdir "$WORKDIR" \
    "$ODT"

echo "✓ Done: $PDF"

Background

The goal is to be able to access and add files to what was originally set up as a local file server, from outside of the LAN. In order to add an additional layer of security, this documentation assumes the use of an intermediate system. That is, a third system to remotely connect to, on the same LAN as the file server, that can then be used to access and manipulate the file server, without having to leave an open VPN connection on the file server itself.

A map of my basic jumpHost set-up

Required Software

• Tailscale: A very simple VPN service, with great apps for MacOS, Windows, Linux, BSD, Android and iOS.
• OpenSSH: For remote access.
• Scp, rSync or both: To remotely move files and directories.
• Curl: Required for installing Tailscale on Linux.

Optional Software

• Terminus: An iOS & Android solution that allows users to save ssh information for easy remote connection.

Install Instructions

Dependencies (SSH, SCP, rSync & Curl)

## Ubuntu
sudo apt install openssh-server curl rsync &&
sudo systemctl enable --now sshd

## Arch
sudo pacman -S openssh scp curl rsync && 
sudo systemctl enable --now sshd

## Fedora
sudo dnf install openssh-server scp curl rsync &&
sudo systemctl enable --now sshd

## FreeBSD
sudo pkg install openssh scp curl rysync ; 
sudo kidload fuse && 
sudo sysrc sshd_enable="YES" &&
sudo service sshd start

## MacOS (Via Homebrew)
brew install openssh scp curl rsync &&
brew install --cask macfuse

Tailscale

Linux

curl -fsSL https://tailscale.com/install.sh | sh

Android or iOS

Install the Tailscale app from the Google Play Store or iOS App Store and login to a Tailscale account, accept any necessary permissions.

MacOS

brew install tailscale

Starting Tailscale

Desktop (Linux, MacOS, Powershell)

sudo tailscale up

After starting the Tailscale process, an authentication link will be printed to the terminal. Follow the URL, sign in to a Tailscale account.

To stop an active VPN tunnel, run:

sudo Tailscale down

A Note on Unattended Use

By default a Tailscale connection will stay open until a user closes the process. Tailscale can also be set to automatically start back up upon a system restart with:

tailscale up --unattended=true

The best resource for use case specific information is Tailscale's documentation, particularly the page on unattended use.

Android or iOS

Tailscale is toggled on an off via the respective phone application. Much the same as on desktop, once the VPN tunnel is open, by default it will remain open indefinitely.

Access

In order to connect to a device remotely all that is required is an open Tailscale tunnel on each device.

SSH Access

Visiting https://login.tailscale.com on any device that you are logged in on, will provide a rundown of all machines authenticated with a given Tailscale account. Here you can easily find IP addresses for remote connection, and copy them to the clipboard. Additionally you can access the IP addresses by opening any Tailscale app, and if running on MacOS, FreBSD or Linux, by running:

sudo tailscale status

Once you have the IP address, the SSH syntax is the same as it would be if you were remotely accessing a device on LAN. By default Tailscale uses port 22 for SSH. It is important to know the username for the device you are accessing via Tailscale's service. However, per standard SSH operation the username part of the equation is irrelevant if you use the same username across machines.

ssh USER@100.XXX.XX.XX # If username is different between the two machines
ssh 100.XXX.XX.XX # If username is the same

A Note on Best Practices & Security

This guide focuses primarily on using an intermediate box to download items via a jump host or intermediate box & then transfer them to a secondary host. This is done out of convenience as much as out of security, as the LAN connection of a stationary server is likely far faster than a remote device that a user can connect with. However using a jump host is also a valid security practice & and OpenSSH has built in functionality to make this process easier and more secure, if a user is attempting to connect to a device like the File Server mentioned in the image at the top of this documentation.

From a remote device, it is generally best practice to use the -J flag in SSH to simply make one connection that connects a user from their remote machine, to the File Server, using the jump host as an intermediate connection point:

ssh -J user@jumpHost user@fileServer

## Or, if using custom ports: 

ssh -J user@jumpHost:port user@fileServer -p port

Moving Files

Both Scp and rSync work to move files from a remote access machine, to a file server. rSync is typically considered more robust, and will work faster for large file transfers. Tailscale also has a file sharing protocol of their own in the Alpha stage of development called, Taildrop.

Scp

scp /path/to/local/file USER@REMOTE:/path/to/remote/directory/ # Basic Syntax

Moving a single file is great for a simple test, but most use cases require moving an entire directory structure. This can be accomplished by adding the -r flag, for recursive transfer, to the beginning of the Scp command:

scp -r /path/to/local/directory USER@REMOTE:/path/to/remote/directory/ # Basic Syntax

rSync

rSync, for most use cases, is a far better option for remote file transfers. The syntax however is nearly identical to using Scp. The only change is that rSync requires the use of a -a flag, for archive transfer, in order to transfer a directory & it's contents.

rsync -a /path/to/local/directory USER@REMOTE:/path/to/remote/directory/

However, where rSync becomes a much more powerful tool, is in the additional functionality it provides. A few of the most helpful options are:

• -a: To move a directory & it's contents
• -v: Verbose (to have the command tell you what it's doing)
• -z: To apply compression to a transfer
• --progress: To show a progress bar for longer transfers.
• --partial: To continue a file transfer that failed.

If a transfer does fail, it can restarted by running rSync with the same parameters and the --partial flag.

rsync -av /path/to/local/directory USER@REMOTE:/path/to/remote/directory/

# Transfer fails :(

rsync -av --partial /path/to/local/directory USER@REMOTE:/path/to/remote/directory/

Important Note

Using both Scp and Rsync, pay very close attention to the placement of forward slashes. In a case like this where the goal is to move the entire directory structure of path/to/local/directory no slash should be included:

/path/to/local/directory not /path/to/local/directory/.

However, in order to ensure the directory structure ends up in /path/to/remote/directory/, this location should include a trailing forward slash:

/path/to/remote/directory/ not /path/to/remote/directory.

Getting Setup to Download Linux ISOs ;)

As a bonus round this documentation will walk through set-up and use of a command line instance of Transmission, as downloading new Linux ISOs via the BitTorrent protocol and storing them on a file server seems to a pretty common use for these servers.

Installation

sudo apt install transmission-cli # Ubuntu
sudo dnf install transmission-cli # Fedora 
sudo pacman -S transmission-cli # Arch
brew install transmission-cli # MacOS (vai Homebrew)

After installation, start the Transmission Daemon.

sudo systemctl enable --now transmission-daemon

Usage

Important Note: This documentation will not provide full usage instructions for Transmission, just enough to get started. More complete documentation can be found on the Arch wiki.

It's helpful to change the default download directory before downloading any torrents:

transmission-remote -w ~/Downloads

Transmission can now download torrent via a .torrent file or a magnet link using the transmission-cli command:

# With a .torrent file
transmission-cli -a /path/to/file

# With a magnet link
transmission-cli -a "MAGNET LINK"

Install Required Packages

We need to first install all packages that are required by BSPWM and are referenced in my BSPWM configuration. Most packages can be installed via the dnf package manger:

sudo dnf upgrade && sudo dnf install xorg-x11-server-Xorg polybar dunst sxhkd bspwm picom feh flameshot redshift xrandr udiskie alacritty pavucontrol brightnessctl xbindkeys

Note: The only packages that are required for BSPWM to function are:

• bspwm
• sxhkd
• xrandr
• X11-server

My polybar configuration will also use a few different scripts that will require API keys stored in a Dropbox folder. Installing and logging into the default Dropbox client is required for some polybar modules to work. In order to install dropbox, you must first download the appropriate rpm package from the Dropbox website & then install with:

sudo dnf install ~/Downloads/DROPBOX-VERSION.rpm

Finally, BSPWM works a lot better with an application launcher of some sort. My configuration is set up to use a custom build of dmenu that will need to be installed. To install we will need to clone the repo via either HTTPS or SSH and then build the application.

cd ~/Documents; git clone git@github.com:makccr/suckless; cd suckless
# or
cd ~/Documents; git clone https://github.com/makccr/suckless.git; cd suckless

Once the repository is cloned, there is an install script located in the root folder, which needs to be run, however in order to build dmenu some X11 development headers to be installed on the system in question:

sudo dnf install libX11-devel libXinerama-devel libXft-devel

./install.sh

Note: It is not necessary to install a custom iteration of demnu. Some amount of time and effort could be saved by simply downloading and installing the dmenu packages in dnf package manger:

sudo dnf install dmenu

Install Configuration Files & Fonts

First use git to clone my dot files repository and create symbolic links for my config and script directories.

cd ~/Documents; git clone git@github.com:makccr/dot; cd
# or
cd ~/Documents; git clone https://github.com/makccr/dot.git; cd

There is a setup script in the .scripts/ folder of this repository as well. It should be noted however, that on a clean install of Fedora Workstation the ~/.config folder will be populated. It is necessary to move these default configuration files somewhere before running the install script.

mv ~/.config ~/Documents/

At this point we can now run the setup.sh script, which will create symbolic links from the ~/Documents/dot directory to the respective locations for all configs. Important: This script must be run from the users home directory (~/).

./~/Documents/dot/.scripts/setup.sh

At this point we can now add the default Fedora configs back into the ~/.config folder:

mv ~/Documents/.config/{*,.*} ~/.config

Lastly we need to install some fonts that polybar will be looking for in it's configuration. The easiest way to do this would be to simply install two font packages from dnf.

sudo dnf install julietaula-montserrat-fonts nerd-fonts-hack

However, this is also a good opportunity to go ahead an install my full collection of fonts:

git clone git@github.com:makccr/fonts; cd fonts; sudo cp -r makc-fonts/ /usr/share/fonts/

Testing

One of the few issues I have run into, is the bspwm.desktop file not being created and put into /usr/share/xessions. This will result in a situation where BSPWM can not be launched from the default login manger. In order to fix this we simply need to create a bspwm.desktop file in /usr/share/xsessions.

Note: Ensuring that xorg-x11-server-Xorg and bspwm packages are installed should prevent this problem from ever taking place, but it's still helpful to know how to fix the issue.

sudo nano /usr/share/xsessions/bspwm.desktop

Add the following text to the .dekstop file and save changes.

[Desktop Entry]
Name=BSPWM
Comment=Binary Space Partitioning Window Manager
Exec=/usr/bin/bspwm
Type=Application

Install Required Packages

#### Arch
sudo pacman -Syu git nodejs npm

#### Fedora
sudo dnf upgrade && sudo dnf install git nodejs npm

#### Ubuntu
sudo apt update && sudo apt upgrade; curl -fsSL https://deb.nodesource.com/setup_current.x | sudo -E bash - && sudo apt install -y nodejs; sudo apt install git npm

#### Free BSD
su -c "pkg update && pkg upgrade && pkg install git node npm"

Clone the Site's Repository

1. Navigate to a suitable location, typically I will use a ~/Documents folder
2. Clone the site from GitHub either via HTTPS or SSH:

git clone git@github.com:makccr/makc.co
or
git clone https://github.com/makccr/makc.co

Install 11ty & Required Plugins

npm install @11ty/eleventy @11ty/eleventy-img airtable dotenv striptags @11ty/eleventy-plugin-rss @11ty/eleventy-plugin-syntaxhighlight; npm install luxon --save-dev

Relocate dotenv API Key

I use the dotenv npm plugin in order to authenticate a connection to Airtable. By default the dotenv plugin expects to find a file called .env in the site's root directory. For obvious reasons this can't be added to the GitHub repository and pushed upstream. My solution is to store the required API key in a Dropbox folder and then manually copy this key into the site's root directory when cloning a new development instance:

cp ~/Dropbox/keys/.env ~/Documents/makccr.github.io/.env

Verify the New Instance is Ready for Development

The easiest way to verify that everything is working properly, is simply to attempt to compile and serve the eleventy site:

npx @11ty/eleventy --serve

If all went well, a local version of the site can now be reached at http://localhost:8080 in any web browser. So long as this command is running the site will continually recompile as changes are made. You can also manually compile the site with:

npx @11ty/eleventy

When ready to commit changes to the site, just use the normal git process for doing so. The site is hosted on Netlify & any changes pushed to the master branch will go live immediately.

git add -A; git commit -m "message"; git push

Helpful Formatting Blocks

New Essay Header

---
title: "TITLE TEXT"
layout: essay.html
date: YYYY-MM-DD
tags: essay 
subject: ["subject1", "subject2"]
---

New Doc Header

---
title: "TITLE"
layout: docs.html
date: YYYY-MM-DD
updated: YYYY-MM-DD
tags: docs
subject: ["subject1", "subject2"]
---

Custom YouTube Embed Code

<div class="youtube">
    <div class="youtube-video-container-16">
        <iframe width="560" height="315"
            src="LINK"
            frameborder="0"
            allow="accelerometer; encrypted-media; gyroscope; picture
-in-picture"
            allowfullscreen >
        </iframe>
    </div>
</div>

This YouTube embed code has additional classes for videos at different aspect ratios, options include:

• youtube-video-container-16
• youtube-video-container-2-1
• youtube-video-container-anamorphic