Snap your Shell scripts !!!

A colleague recently talked me into buying one of these nifty HDMI to USB video capture dongles that allows me to try out my ARM boards attached to my desktop without the need for a separate monitor. Your video output just ends up in a window on your desktop … this is quite a feature for just 11€

The device shows up on Linux as a new /dev/video* device when you plug it in, it also registers in pulseaudio as an audio source. To display the captured output a simple call to mplayer is sufficient, like:

mplayer -ao pulse tv:// -tv driver=v4l2:device=/dev/video2:width=1280:height=720

Now, you might have other video devices (i.e. a webcam) attached to your machine and it will not always be /dev/video2 … so we want a bit of auto-detection to determine the device …

Re-plugging the device while running dmesg -w shows the following:

usb 1-11: new high-speed USB device number 13 using xhci_hcd
usb 1-11: New USB device found, idVendor=534d, idProduct=2109, bcdDevice=21.00
usb 1-11: New USB device strings: Mfr=1, Product=2, SerialNumber=0
usb 1-11: Product: USB Video
usb 1-11: Manufacturer: MACROSILICON
uvcvideo: Found UVC 1.00 device USB Video (534d:2109)
hid-generic 0003:534D:2109.000C: hiddev2,hidraw7: USB HID v1.10 Device [MACROSILICON USB Video] on usb-0000:00:14.0-11/input4

So our vendor id for the device is 534d … this should help finding the correct device from Linux’ sysfs … lets write a little helper:

VIDEODEV=$(for file in /sys/bus/usb/devices/*/idVendor; do 
  if grep -q 534d $file 2>/dev/null; then 
    ls $(echo $file| sed 's/idVendor/*/')/video4linux; 
  fi;
done | sort |head -1)

Running this snippet in a terminal and echoing $VIDEODEV now puts out “video2”, this is something we can work with … lets put both of the above together into one script:

#! /bin/sh

VIDEODEV=$(for file in /sys/bus/usb/devices/*/idVendor; do 
  if grep -q 534d $file 2>/dev/null; then 
    ls $(echo $file| sed 's/idVendor/*/')/video4linux; 
  fi;
done | sort |head -1)

mplayer -ao pulse tv:// -tv driver=v4l2:device=/dev/$VIDEODEV:width=1280:height=720

Making the script executable with chmod +x run.sh (i have called it run.sh) and executing it as ./run.sh now pops up a 720p window showing the screen of my attached Raspberry Pi.

Video works now, lets take a look at how we can get the audio output too.
First we need to find the correct name for the pulseaudio source, again based on the Vendor Id:

AUDIODEV=$(pactl list sources | egrep 'Name:|device.vendor.id.' | grep -B1 534d | head -1 | sed 's/^.*Name: //')

Running the above and then echoing $AUDIODEV returns alsa_input.usb-MACROSILICON_USB_Video-02.analog-stereo so this is our pulse source we want to capture and play back to the default audio output, this can be easily done with a pipe between two pacat commands, one for record (-r) and one for playback (-p) like below:

pacat -r --device="$AUDIODEV" --latency-msec=1 | pacat -p --latency-msec=1 

Now playing a video with audio on my Pi while running the ./run.sh script in one terminal and the pacat pipe in a second one gives me both, video and audio output …
To not have to use two terminals we should rather merge the pacat, auto detection and mplayer commands into one script … since both of them are actually blocking, we need to fiddle a bit by putting pacat into the background (by adding a & to the end of the command) and telling our shell to actually kill all subprocesses (even backgrounded ones) that were started by our script when we stop it with the following trap command:

pid=$$
terminate() {
  pkill -9 -P "$pid"
}
trap terminate 1 2 3 9 15 0

So lets merge everything into one script, it should look like below then:

#! /bin/sh

pid=$$
terminate() {
  pkill -9 -P "$pid"
}
trap terminate 1 2 3 9 15 0

VIDEODEV=$(for file in /sys/bus/usb/devices/*/idVendor; do 
  if grep -q 534d $file 2>/dev/null; then 
    ls $(echo $file| sed 's/idVendor/*/')/video4linux; 
  fi;
done | sort |head -1)

AUDIODEV=$(pactl list sources | egrep 'Name:|device.vendor.id.' | grep -B1 534d | head -1 | sed 's/^.*Name: //')

pacat -r --device="$AUDIODEV" --latency-msec=1 | pacat -p --latency-msec=1 &

mplayer -ao pulse tv:// -tv driver=v4l2:device=/dev/$VIDEODEV:width=1280:height=720

And this is it, executing the script now plays back video and audio from the dongle …

Collecting all the above info to create that shell script took me the better part of a Sunday afternoon and I was figuring that everyone who buys such a device might hit the same pain, so why not package it up in a distro agnostic way so that everyone on Linux can simply use my script and does not have to do all the hackery themselves … snaps are an easy way to do this and they are really quickly packaged as well, so lets do it !

First of all we need the snapcraft tool to quickly and easily create a snap and use multipass as build environment:

sudo snap install snapcraft --classic
sudo snap install multipass

Now lets create a workdir, copy our script in place and let snapcraft init create a template file as a boilerplate:

$ mkdir hdmi-usb-dongle
$ cd hdmi-usb-dongle
$ cp ../run.sh .
$ snapcraft init
Created snap/snapcraft.yaml.
Go to https://docs.snapcraft.io/the-snapcraft-format/8337 for more information about the snapcraft.yaml format.
$

We’ll edit the name in snap/snapcraft.yaml, change core18 to core20 (since we really want to be on the latest base), adjust description and summary, switch grade: to stable and confinement: to strict … Now that we have a proper skeleton, lets take a look at the parts: which tells snapcraft how to build the snap and what should be put into it … we just want to copy our script in place and make sure that mplayer and pacat are available to it … To copy a script we can use the dump plugin that snapcraft provides, to make sure the two applications our script uses get included we have the stage-packages: property, the parts: definition should look like:

parts:
  copy-runscript: # give it any name you like here
    plugin: dump
    source: . # our run.sh lives in the top level of the source tree
    organize:
      run.sh: usr/bin/run # tell snapcraft to put run.sh into a PATH that snaps do know about
    stage-packages:
      - mplayer
      - pulseaudio-utils

Now we can just call snapcraft while inside the hdmi-usb-dongle dir:

$ snapcraft
Launching a VM.
Launched: snapcraft-my-dongle
[...]
Priming copy-runscript 
+ snapcraftctl prime
This part is missing libraries that cannot be satisfied with any available stage-packages known to snapcraft:
- libGLU.so.1
- libglut.so.3
These dependencies can be satisfied via additional parts or content sharing. Consider validating configured filesets if this dependency was built.
Snapping |                                                                                                                   
Snapped hdmi-usb-dongle_0.1_amd64.snap

OOPS ! Seems we are missing some libraries and snapcraft tells us about this (they are apparently needed by mplayer)… lets find where these libs live and add the correct packages to our stage-packages: entry … we’ll install apt-file for this which allows reverse searches in deb packages:

$ sudo apt install apt-file
[...]
$ sudo apt-file update
$ apt-file search libGLU.so.1                               
libglu1-mesa: /usr/lib/x86_64-linux-gnu/libGLU.so.1
libglu1-mesa: /usr/lib/x86_64-linux-gnu/libGLU.so.1.3.1
$ apt-file search libglut.so.3
freeglut3: /usr/lib/x86_64-linux-gnu/libglut.so.3
freeglut3: /usr/lib/x86_64-linux-gnu/libglut.so.3.9.0

There we go, lets add libglu1-mesa and freeglut3 to our stage-packages:

    stage-packages:
      - mplayer
      - pulseaudio-utils
      - libglu1-mesa
      - freeglut3

If we now just call snapcraft again, it will re-build the snap for us and the warning about the missing libraries will be gone …

So now we do have a snap containing all the bits we need, the run.sh script, mplayer and pacat (from the pulseaudio-utils package). We also have made sure that mplayer finds the libs it needs to run, now we just need to tell snapcraft how we want to execute our script. To do this we need to add an apps: section to our snapcraft.yaml:

apps:
  hdmi-usb-dongle:
    extensions: [gnome-3-38]
    command: usr/bin/run
    plugs:
      - audio-playback   # for the use of "pacat -p" and "pactl list sources"
      - audio-record     # for the use of "pacat -r"
      - camera           # to allow read access to /dev/videoX
      - hardware-observe # to allow scanning sysfs for the VendorId 

To save us from having to fiddle with any desktop integration, there are the desktop extensions (you can see which extensions exist with the snapcraft list-extensions command), since we picked base: core20 at he beginning when editing the template file, we will use the gnome-3-38 extension with our snap. Our app should execute our script from the place we have put it in with the organize: statement before so our command: entry points to usr/bin/run and to allow the different functions of our script we add a bunch of snap plugs that I have explained inline above. Now our snapcraft.yaml looks like below:

name: hdmi-usb-dongle
base: core20
version: '0.1'
summary: A script to use a HDMI to USB dongle
description: |
  This snap allows to easily use a HDMI to USB dongle on a desktop
grade: stable
confinement: strict

apps:
  hdmi-usb-dongle:
    extensions: [gnome-3-38]
    command: usr/bin/run
    plugs:
      - audio-playback
      - audio-record
      - camera
      - hardware-observe

parts:
  copy-runscript:
    plugin: dump
    source: .
    organize:
      run.sh: usr/bin/run
    stage-packages:
      - mplayer
      - pulseaudio-utils
      - libglu1-mesa
      - freeglut3

And this is it … running snapcraft again will now create a snap with an executable script inside, you can now install this snap (because it is a local snap you need the –dangeours option), connect the interface plugs and run the app (note that audio-playback automatically connects on desktops, so you do not explicitly need to connect it) …

$ sudo snap install --dangerous hdmi-usb-dongle_0.1_amd64.snap
$ sudo snap connect hdmi-usb-dongle:audio-record
$ sudo snap connect hdmi-usb-dongle:camera
$ sudo snap connect hdmi-usb-dongle:hardware-observe

When you now run hdmi-usb-dongle you should see something like below (if you have a HDMI cable connected with a running device you will indeed not see the test pattern):

This is great, everything runs fine, but if we run this on a desktop an “Unknown” icon shows up in the panel … it is also annoying having to start our app from a terminal all the time, so lets turn our snapped shell script into a desktop app by simply adding a .desktop file and an icon:

$ pwd
/home/ogra/hdmi-usb-dongle
$ mkdir snap/gui

We’ll create the desktop file inside the snap/gui folder that we just created, with the following content:

[Desktop Entry]
Type=Application
Name=HDMI to USB Dongle
Icon=${SNAP}/meta/gui/icon.png
Exec=hdmi-usb-dongle
Terminal=false
Categories=Utility;

Note that the Exec= line just uses our command from the apps: section in our snapcraft.yaml
Now find or create some .png icon, 256×256 is a good size (I tend to use flaticon.com to find something that fits (do not forget to attribute the author if you use downloaded icons, the description: field in your snapcraft.yaml is good for this)) and copy that icon.png into snap/gui …

Re-build your snap once again, install it with the --dangerous option and you should now find it in your application overview or menu (if you do not use GNOME).
Your snapped shell script is done, congratulations !

You could now just upload it to snapcraft.io to allow others to use it … and here we are back to the reason for this blog post … as I wrote at the beginning it took me a bit of time to figure all the commands out that I added to the script … I’m crazy enough to think it might be useful for others, even though this USB dongle is pretty exotic hardware, so I expected it to probably find one or two other users for whom it might be useful and I created https://snapcraft.io/hdmi-usb-dongle

For snap publishers the snapcraft.io page offers the neat feature to actually see your number of users. I created this snap about 6 weeks ago, lets see how many people actually installed it in this period:

Oh, seems I was wrong, there are actually 95 (!) people out there that I could help with packaging my script as a snap package !! While indeed the majority of users will be on Ubuntu given that snaps are a default package tool there, even among these 95 users there are a good bunch of non Ubuntu systems (what the heck is “boss 7” ??):

So if you have any useful scripts lying on your disk, even for exotic tasks, why not share them with others ? As you can see from my example even scripts to handle exotic hardware quickly find a lot of users around the world and across different distros when you offer them in the snap store … and do not forget, snap packages can be services, GUI apps as well as CLI apps, there are no limits in what you can package as a snap !