Tuesday, January 30, 2024

Editing PDF's in Windows and Linux

One of the main promises of digitization is “going paperless” — using digital (instead of paper) documents for record-keeping and other management processes. To a large extent this has come true, save for one stumbling block: signing documents. While digital signing solutions do exist and are getting more popular, there are still plenty of times when we have to do this ridiculous song-and-dance:

  1. Receive a PDF document by email;
  2. Print the document;
  3. Sign the document;
  4. Scan the document;
  5. Email back the scanned signed document.

Surely there must be a better way.

On Windows, Adobe’s Acrobat Reader includes tools for filling a PDF form, including the option of inserting a scanned signature. The interface can be a little awkward, especially in edge cases such as filling multiple signature fields, but with a little practice it can be made to work quite well.

Unfortunately, Acrobat Reader isn’t available for Linux. You can always run the Windows version in a virtual machine, but that’s rather compute-intensive; another option is installing Acrobat with Wine, but support isn’t perfect at the moment.

What about third-party tools? There are several options, most with their particular downsides:

  • GIMP is an image editor tool with PDF support. It does a great job of converting PDF documents to images, which can then be freely edited — but exporting simply outputs the imported data as images, which degrades document quality and drastically increases file size.
  • LibreOffice Draw can edit PDF files, including the ability to add (signature) images to a document. However it often has problems rendering fonts correctly, leading to format corruption.
  • Okular has perfect rendering support and powerful annotation tools, including the ability to add images to documents as “stamps” — but saving these changes in a format visible by other tools requires printing the modified document to file, which degrades quality and increases file size, sometimes by as much as four times.
  • QPDF is a command-line tool and C++ library for manipulating PDF files. In the case of multi-page documents where only a few have to be edited, it can be used to attenuate the file size issues of the other options, by splicing unmodified pages from the original document with modified ones generated by other tools.
  • Scribus is a desktop publishing application that can import and export PDF files. It can be used to import a document, add text and image fields as needed, then export it back to PDF. The exported documents look perfect, but they are huge — in my experiments a 1MB document ballooned to over 30 MB after exporting.

As of this writing, the one tool that worked perfectly for me is Xournal++, “a handwriting notetaking software with PDF annotation support”. PDF rendering support is excellent, and the UI is about as good as Acrobat’s. The only warning is that you should definitely install the latest stable release, since even slightly outdated versions tend to be a little buggy. This is especially true for Ubuntu LTS and derivatives which tend to trail behind upstream repositories.

To install the latest stable release of Xournal++ to Ubuntu, open a terminal session and enter the commands below:

sudo add-apt-repository ppa:apandada1/xournalpp-stable
sudo apt update
sudo apt install xournalpp

For other distributions, see the project page for details.

Once Xournal++ is installed, PDF files can be opened from the main menu by selecting File -> Open, or by using the Ctrl+O shortcut. Common editor options include:

  • Add Image: Tools -> Image or Shift+Ctrl+I — then click on the location where you want to add an image. A file selection dialog will automatically open. Once the image is added it can be freely moved and scaled. If you want to scale an image while keeping its proportion, click and drag one of the corners of the image bounding box; to scale without maintaining proportion, click and drag from the middle of one of the sides.
  • Add Text: Tools -> Text or Shift+Ctrl+T — then click on the location where you want to add a text box. Font face, size and effects (i.e. italic and / or bold) can be selected from a tool bar drop down. The text box expands in response to the amount of text; line breaks can be added manually.
  • Select Rectangle: Tools -> Select Rectangle or Shift+Ctrl+R — allows previously added elements to be selected, moved and copied / pasted (with Ctrl+C / Ctrl+V).

Notice that when a tool is selected, it remains active until something else is selected — for example, if Add Image is selected, the file selection dialog will open every time you click the document. It’s good practice to manually revert to Select Rectangle every time a new element is added to the document, to avoid surprises.

After completing the desired edits, File -> Export as PDF can be used to export the whole document with the changes. To export just the modifed page(s), you can use File -> Print, select Print to File and enter the desired page(s). Also remember to set the destination file path, which defaults to ~/Documents/output.pdf.

Sunday, January 28, 2024

Installing and using the Arduino Command-Line Interface (CLI)

The Arduino line of microcontrollers has become a de facto standard in the maker world. One of its main advantages is the Arduino IDE, which makes it very easy to program and debug boards.

However, in more complex projects, it’s often desirable to centralize development around a common toolset, including a general-purpose text editor or IDE. In this case a command-line interface is necessary so that Arduino sketches can be automatically compiled and uploaded from build scripts, instead of manually through the Arduino IDE.

In the past, the Arduino IDE supported command-line options for automatically running tasks such as compiling or uploading a sketch to a board. However, more recently these functionalities have been moved to a new application, the Arduino Command-Line Interface (CLI).

Installing the Arduino CLI can be easily done from the terminal. To install the tool under ~/.local/bin, make sure you have curl installed, then run the commands below:

ARDUINO_PATH="$HOME/.local/bin"
mkdir -p $ARDUINO_PATH
curl -fsSL "https://raw.githubusercontent.com/arduino/arduino-cli/master/install.sh" | BINDIR=$ARDUINO_PATH sh

For convenience, make sure that $HOME/.local/bin is in your $PATH environment variable. If not, add the following line to ~/.bashrc:

export PATH="$PATH:$HOME/.local/bin"

Then either start a new terminal session or run the command above in the current session to update the $PATH.

As initially set up, Arduino CLI is unable to communicate with any board. This requires installing “cores”, which contain information and specific tools for specific board architectures. First, download the list of available cores with:

arduino-cli core update-index

In order to know which core to install, connect your Arduino board to your computer, then run the command below:

arduino-cli board list

For example, with a single Arduino UNO connected, the output will be something like this:

Port         Protocol Type              Board Name  FQBN            Core
/dev/ttyACM0 serial   Serial Port (USB) Arduino Uno arduino:avr:uno arduino:avr

Now run the command below:

arduino-cli core install $CORE

Replacing $CORE with the value in the Core column from the output of arduino-cli board list — in the case above, arduino:avr.

If your code uses any extra libraries, they also need to be installed. For example, to install the TimerInterrupt library, enter:

arduino-cli lib install "TimerInterrupt"

Once your environment is properly set up, a sketch can be compiled with:

arduino-cli compile --fqbn $FQBN --warnings all $PATH_SKETCH

Where $FQBN is the Fully-Qualified Board Name (arduino:avr:uno in the example above). If you have your board connected to your computer, finding the FQBN can be automated with:

arduino-cli compile \
  --fqbn $(arduino-cli board list | grep -oEi 'arduino:[^ ]+:[^ ]+') \
  --warnings all \
  $PATH_SKETCH

In contrast to the Arduino IDE, the Arduino CLI does not automatically compile a sketch before upload, so make sure to run this command following any changes to the code. Also notice that we set option –warnings all to make sure we’re notified in case of any compilation warnings. This is important since warnings can often be indicative of programming mistakes, and it’s generally good practice to resolve them whenever possible.

Finally, a sketch can be uploaded to a board with:

arduino-cli upload -p $PORT --fqbn $FQBN $PATH_SKETCH

Where $PORT and $FQBN can be found from the output of arduino-cli board list as shown above. This can be automated with a little shell scripting as below:

arduino-cli upload \
  $(arduino-cli board list | perl -ne 'm/([^ ]+).+(arduino:[^ ]+:[^ ]+)/ && print "-p $1 --fqbn $2"') \
  "$PATH_SKETCH"

The above commands can be added to shell scripts for easy reuse:

#!/bin/bash

# arduino-compile.sh

arduino-cli compile \
  --fqbn $(arduino-cli board list | grep -oEi 'arduino:[^ ]+:[^ ]+') \
  --warnings all \
  "$1"
#!/bin/bash

# arduino-upload.sh

arduino-cli upload \
  $(arduino-cli board list | perl -ne 'm/([^ ]+).+(arduino:[^ ]+:[^ ]+)/ && print "-p $1 --fqbn $2"') \
  "$1"

Create the scripts under ~/.local/bin, then give them execute permission with:

chmod u+x $HOME/.local/bin/arduino-compile.sh
chmod u+x $HOME/.local/bin/arduino-upload.sh

The scripts can then be used as below:

arduino-compile.sh "$PATH_SKETCH"
arduino-upload.sh "$PATH_SKETCH"

Tuesday, January 23, 2024

A Selection of Single-Board Computers and Mini PC's for Maker Projects

Many maker projects require a central computer unit to process sensor streams, run analysis and decision-taking algorithms, control actuators and provide an interface to users, among other tasks. Different projects demand varying levels of mobility, power consumption and processing power, with budget constraints being an additional consideration factor; in the end most options fall in either the category of Single-Board Computer (SBC) or mini PC.

The list below provides a variety of computing platform alternatives, with considerations on what projects they match best.

Raspberry Pi 4

The Raspberry Pi is a line of ARM-based SBC’s. In terms of cost-effectiveness it strikes a balance between more economical (but also limited) micro-controllers and more powerful (but also expensive) mini PC’s. Although recently superseded by the Raspberry Pi 5, the Raspberry Pi 4 may still be preferred for many automation tasks, owing to its lower power consumption and compatibility with existing accessories.

When buying a Raspberry Pi, it’s important to keep in mind that additional parts will be required for its operation. At a minimum, a Power Source Unit (PSU) and a micro SD card are needed, and a case is also highly recommended. The Raspberry Pi OS image can be pre-formatted for automatic connection to a Wi-Fi network, allowing for headless operation from the get-go; still it’s also recommended to have an HDMI adapter at hand, in case connection to a display is necessary.

nVIDIA Jetson Nano

the Jetson Nano is an SBC combining an ARM CPU and an nVIDIA 128-core Maxwell GPU. It provides a cost-effective solution for edge AI projects, especially those combining AI-based Computer Vision and robotics.

The Jetson Nano computing module is usually sold alongside its custom expansion board, but not much else. It’s recommended to also acquire a complementary kit including a case, power source, micro SD card and Wi-Fi / Bluetooth card, in order to make the system more usable.

Asus PN53

The mini PC form-factor got a huge boost when Intel launched the NUC line, but it was arguably perfected by ASUS and it’s "P*" line. These are great “workhorse” machines, for when raw CPU power is the overriding requirement — for example, in mobile robotics applications. The PN53 is a recent example of the line. It’s based on AMD’s Ryzen 7 7735HS APU; launched in January 2023, it contains 8 CPU cores, which between them can run 16 threads.

When buying a mini PC, check whether it’s being sold “barebones”, meaning that it comes without memory or storage — which have to be acquired separately. For most maker applications, 16GB of RAM and 250GB of storage will do nicely, which tend to be on low end of the cost spectrum.

Sunday, January 21, 2024

FFmpeg Quick Reference

FFmpeg is a powerful command-line editing tool, but sometimes it can be hard to track down how to perform certain operations. The list below is a quick command reference for common tasks:

Convert video to MP4

ffmpeg -i $INPUT_PATH -vcodec h264 -acodec mp2 $OUTPUT_PATH

Crop video

ffmpeg -i $INPUT_PATH -filter:v "crop=$WIDTH:$HEIGHT:$X:$Y" $OUTPUT_PATH

Remove audio

ffmpeg -i $INPUT_PATH -c copy -an $OUTPUT_PATH

Cut video

ffmpeg -i $INPUT_PATH -ss $START_TIMESTAMP -t $DURATION -c:v libx264 $OUTPUT_PATH

Where $START_TIMESTAMP and $DURATION are strings in the hh:mm:ss.s format.

Scale video proportionally by width

ffmpeg -i $INPUT_PATH -filter:v scale=$WIDTH:-1 -c:a copy $OUTPUT_PATH

Where $WIDTH is the output video width.

Scale video proportionally to 1080p, add black bands as appropriate to fill dimensions

ffmpeg -i $INPUT_PATH \
  "scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,setsar=1" \
  -c:a copy $OUTPUT_PATH

Scale video proportionally to 800 pixels width, set 15 FPS and average bit rate of 2 Mb

ffmpeg -i $INPUT_PATH -b:v 2M -maxrate 2M -bufsize 1M -vf "fps=15,scale=800:-1" $OUTPUT_PATH

git Quick Reference

The git versioning system is the de-facto standard for version control in the FOSS world and much of the software industry. However, its command-line interface can sometimes be hard to figure out, even for seasoned users. The list below is a quick command reference for common tasks:

Pulling Changes from Remote Repository

Pull repository data

git pull --rebase

Adding the --rebase option ensures that any local commits not yet pushed to the remote repo will be placed after any new pulled from it. This makes it possible to then push these commits without getting into any versioning conflicts.

Pull remote branch

git fetch origin remote_branch_name
git checkout -b remote_branch_name origin/remote_branch_name

Pushing Changes to Remote Repository

Push current branch to remote repository

git push

Push locally-created project into repository

git push --set-upstream <URL> master

Undoing Changes

Create a copy of a file’s previous version

git show HEAD^:$path_to_file > $path_to_copy

Unstage added files before commit

git reset HEAD

Restore a deleted file

Find the last commit that affected the given path. As the file isn’t in the HEAD commit, this commit must have deleted it.

git rev-list -n 1 HEAD -- <file_path>

Then checkout the version at the commit before, using the caret (^) symbol:

git checkout <deleting_commit>^ -- <file_path>

Or in one command, if $file is the file in question.

git checkout $(git rev-list -n 1 HEAD -- "$file")^ -- "$file"

Wipe changes since last commit

git reset --hard

Remove untracked files / directories

git clean -f -d

Rollback last commit, returning changes to uncommitted state

git reset HEAD^

Tag Management

List tags

git tag

Create tag

git tag -a tag-name -m "tag description"

Push repository data with tags

git push --follow-tags origin master

Checkout tag

git checkout -b branch-name tag-name