Current focus: EndSoftwarePatents.org: wiki, mailing list, news, donate

Portrait photo of Ciaran

Some commands I like

Menu


But first... My favourite software packages

These are software packages that make my life easier. I use the GNU/Linux operating system, but most or all of these are available for other operating systems too.

Adding Ogg vorbis comments

Here's a command to add a comment to an Ogg Vorbis audio file:

vorbiscomment -a Ciaran_O\'Riordan_-_End_Software_Patents_-_LibrePlanet_2009.ogg -t "comment=Transcript: http://groups.fsf.org/index.php/Ciaran_O'Riordan_(LP09)"

Making oggs directly from CD

Convert track 1 of a CD to ogg via pipe:

cdparanoia 1-1 - | oggenc -q 3 -o track-01.ogg - ; done

And to do that for a CD of 20 tracks:

for i in `seq -w 1 20`; do cdparanoia ${i}-${i} - | oggenc -q 3 -o track-${i}.ogg - ; done

OCR'ing a .pdf or .ps

gs -sPAPERSIZE=a4 -sDEVICE=pnmraw -r300 -dNOPAUSE -dBATCH -sOutputFile=- -q input.pdf | ocrad > output.text

There's an article about a more thorough method here: How To Extract All Text From PDFs (Including Text In Images).

Replacing spaces in filenames with underscores

Here's a command which operates on all .pdf files in the current directory. It replaces any spaces in the filenames with underscores.

find . -maxdepth 1 -iname '*.pdf' | while read n; do mv "$n" `echo "$n" | sed 's/ /_/g'`; done

Reading lists of words from files

I have two files with lists of words, each has one word per line, and I want to run a command based on all combinasions of those two lists. For a practical example, there's a website that posts videos each day, but they don't provide a convenient way to download them. The file names are different each day, but many use a recurring set of words used to form the file names, so if I try all those combinations, I'll get a few videos each day.

cat prefixes | while read p; do cat suffixes | while read s; do wget -c http://videos.gmanews.tv/2010a/03/${p}_030710_${s}.flv; done ; done

In this example, I'll also have to change the date (mmddyy).

Playing audio at increased speeds

To play a file at 1.7 times normal speed with mplayer:

This increases the playback speed without increasing the pitch.

Playing video at increased speeds

To play a video at increased speed, while keeping the audio and video in sync, you have to launch mplayer with the following command, and then use the keys "]", "}", "[", and "{".

If you set the initial speed to something other than 1, the video will be out of sync. If you launch mplayer without the scaletempo option, when you increase the speed you'll change the pitch of the audio.

More info here.

Math on the bash command line

This is a command I used to make a HTML table so I could see what the timestamp on my audio file turns into when I'm listening with the speed increased by 70%. (See mplayer speed commands.)

for i in `seq 1 100`; do echo \<tr\>\<td\>$((${i} / 60)):$((${i} % 60))\</td\>\<td\>$(( $(( $((${i} * 10)) / 17)) / 60)):$(( $(( $((${i} * 10)) / 17)) % 60))\</td\>\</tr\>; done

Playing audio/video at more than 100% volume

If you've audio or video files where the volume is too low, mplayer can amplify them with this command:

mplayer -softvol -softvol-max 1000 file.ogg

When you play the file, the volume will be at the normal (low) level, but by hitting * you can increase the volume to 1000% of the normal maximum. To decrease the volume, hit /.

Run a program in another language

If you want to run, say, GIMP in Spanish, you can lauch it with this command from a shell command line:

LANGUAGE=es_ES gimp

For some commands such as "date", you use "LANG" instead and you can only use the languages of "locales" that are installed on your system. To see the list of locales, type:
locale -a

LANG=nl_BE.utf8 date

And if you want to specify a list of preferences, there's info in the glibc manual. This is for if you want designate, say, Dutch as your primary language, and French as your second language. If you only set one preference, then you'll always get the default (English) when something isn't translated to your preferred language.

Extract audio from a video (mp3 from flv)

Here's the command I used to get the audio from a .flv video:

mplayer rozemarijn.flv -dumpaudio -dumpfile rozemarijn.mp3

When mplayer is given the dumpaudio command, it dumps an exact copy of the audio data that's in the original file. No conversion is done and no data is lost, so you can do this infinitely without degrading the audio. This also means that you don't choose the format of the resulting audio file. In this case, I knew the audio in the .flv file was in the MP3 format, so I gave the output file a name ending in ".mp3". I could have called the output file "rozemarijn.wav" or "rozemarijn.ogg", but that wouldn't change anything. The audio would still be in MP3 format.

Seeing what URLs are being accessed

The following are suggestions for how to see what URLs your computer is accessing. This can be useful, for example, if you can see a video in your browser and you want to download it directly. I haven't gotten to try these commands out yet. They were suggested in a discussion thread on LWN.net (Thanks!).

A forgotten ffmpeg command

Hmm, I don't remember exactly what this does:

ffmpeg -i filename -ab 56 -ar 44100 -b 500 -s 320x240 filename.mpg

It might have something to do with Rockbox's instructions for shrinking videos.

Here's another. It's changes the size to 176x144, discards the audio (-an), uses 15 frames per second and changes the video codec.

avconv -i inputfile.flv -s 176x144 -an -vcodec h263 -ac 1 -ar 8000 -r 15 -ab 32 -y outputfile.3gp

How to crop a video to remove black borders with mplayer

There are three ways. First is the manual way. This works 100% of the time. The second and third ways require less work but I find them too unreliable.

1. The manual way

First, make a file in the directly ~/.mplayer/ (the filename isn't important, we'll call ours crop-keys) and put this in the file:

u change_rectangle 1 -4    # "1 -4": shrink box, moving bottom line UP
l change_rectangle 0 -4    # "0 -4": shrink box, move right-edge to LEFT
d change_rectangle 1  4    # "1  4": enlarge box, move the bottom DOWN
r change_rectangle 0  4    # "0  4": enlarge box, move right-edge to RIGHT
U change_rectangle 3 -4    # "3 -4": move box UPwards
L change_rectangle 2 -4    # "2 -4": move box to LEFT
D change_rectangle 3  4    # "3  4": move box DOWNwards
R change_rectangle 2  4    # "2  4": move box to RIGHT

Then when you watch your video with the next command, there'll be a white cropping rectangle along the very edge of the screen.

mplayer -vf rectangle -input conf=crop-keys yourvideofile.ogv

So for example, if you're just cropping the top and bottom, you'll hit u a few times, then D, then u again and D again until you get the box where you want it. When you've put the cropping rectangle around the part of the picture you want to keep, exit the video. On the terminal, you'll see a list of numbers. The last line will show your crop coordinates as four colon-separated numbers like this:

A:  21.6 V:  21.6 A-V:  0.000 ct:  0.083   0/  0  0%  0%  0.2% 0 0 
rectangle: -vf rectangle=146:104:20:20 

Stick those coordinates (146:104:20:20) into your command when you watch the video:

mplayer -vf crop=146:104:20:20 yourvideofile.ogv

And that's it. You can resize the screen or start it in fullscreen mode (-fs) and you'll only see the part of the screen that you selected.

2. The semi-automatic way

Mplayer can try to guess the desired crop coordinates with this command:

mplayer -vf cropdetect=round:16  yourvideofile.ogv

Exit the video at any point where the edges of the screen are obvious. E.g. don't exit at a point where the whole screen is black such as during the title credits. You'll see the four colon-separated coordinates on the terminal and you can use them as described above.

3. The fully automatic way

Sometimes (rarely) the whole thing can be done automatically by a command like this, just tweak the number:

mplayer -panscan 0.8 yourvideofile.ogv

Setting up Apache+MySQL+PHP (on Debian GNU/Linux)

Here's what I did to get the necessary Debian packages working to allow me to install MediaWiki 1.17.0 from source. There's a MediaWiki package in Debian but it was out of date. Further down on this page there's my notes about the installation and setup of MediaWiki itself.

Ok, that's Apache working. PHP next.

That's PHP working.

MySQL didn't require any setup. I set a "root" password, but I think that was optional.

Setting up MediaWiki (with the above software)

With Apache, MySQL, and PHP install (per above), here's what I then did to get MediaWiki 1.17.0.

That was pretty easy. I'm surprised that MySQL required no attention at all.

While looking into caching and performance I found people suggesting the following:

Mplayer gives errors - use alternative backends

I got error messages when reading MP3s with Mplayer. With a 2012 version, the error message was something like:

And when I downgraded to a 2010 version of mplayer, I got a different error:

Each repeated hundreds of times.

The solution seems to be (not tested yet) to tell mplayer to use another backend. mplayer -ac help will show the available backends, and you can use another with the command switch -afm ffmpeg (to use the "ffmp3" codec/backend.

Convert a colour pdf to greyscale (black and white)

Very simple with Ghostscript:

On the page where I found that command, the person also added -dCompatibilityLevel=1.4 but I don't see any reason and the command works without this.

I haven't found much good documentation of these Ghostscript options but their ps2pdf page contains some info. And there's some info about detecting if a pdf contains colours and converting it to greyscale.

Convert PDF to HTML or ODT

Short answer: After hours in January 2014, I found there's no high-quality solution. The best option is to try pdftohtml and Abiword and see which gives the best results for your PDF. Here's an overview of all the options:

pdftohtml
Convert first to HTML with pdftohtml, which is part of the Poppler package. On Debian systems, the package to install is poppler-utils. This makes a HTML document, and you can open it in LibreOffice and then save it as ODT. The command line options aren't documented, and their effects are inconsistent, so for best results you have to test each option in each combination. On the document I tried, I got best HTML results with -c -s -nomerge -noframes but then when I opened it in LibreOffice every page was grey so I had to re-test each combination with LibreOffice.
(Results: Strange formatting, but if you want an editable document this might be good enough.)
Abiword
Abiword can open PDF documents. Then just save as ODT.
(Results: Strange formatting, but if you want an editable document this might be good enough.)
LibreOffice
LibreOffice has a "PDF import" extension. By default, LibreOffice then opens PDFs in LibreOffice Draw, with the result that you can't save it as ODT. To open the PDF in LibreOffice Writer, open LibreOffice Writer, use File->Open from the menu, in the "File types" select "PDF (Writer)" then select your PDF. The problem is, the text isn't normal text. I can't describe what it is, but it's completely impractical for editing. On some systems you have to install the "PDF import" extension separately - on Debian, it's the package libreoffice-pdfimport.
(Results: Not much use.)
Karbon
Karbon (part of Calligra Office / Koffice) can import PDF's, but can't export to ODT. Same problem as LibreOffice.
(Results: Not much use.)
MuPDF
Another option might be MuPDF. A few web pages suggest is has a "html output" mode, but I can't find the command or the option necessary.
(Results: No use.)
KWord / Calligra Words
KWord (aka Calligra Words) had a PDF import feature but it was removed some time before 2010.
(Results: No use.)

While loop checking when a file changes via inotifywait

while inotifywait -e modify FILENAME; do make; done

Compressing speech

To make an mp3, LAME is probably the best software. There's a command line option specifically for compressing voice audio:

lame --preset voice inputfile.mp3 outputfile.mp3

But I once got a report once from an iPad user who couldn't play two mp3's I made with that option, so I used the following options and got a file that was the same size and the quality seemed at least as good:

lame -V 7 --vbr-new -h -q 2 -a inputfile.mp3 outputfile.mp3

I used -q 2 because I read somewhere that lame didn't work right with 1 or 0.

I read elsewhere that --preset voice is equivalent to:

--resample 24 --lowpass 12 --noshort yes -mm -b56

I read some documentation about these options and decided that I don't understand them, that it's easy to make broken files by choosing the wrong settings, and that LAME will probably choose some reasonable setting anyway (probably based on my -V and -q settings). So it's best to leave them out. -mm is the same as -a.


(Go: homepage of Ciarán O'Riordan)

Valid XHTML 1.1! Powered by GNU Made with Emacs Join FSF! (I'm member #8)

(I made the Emacs and GNU buttons. Feel free to do with them whatever you like.)

© Copyright 2014 Ciarán O'Riordan. Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved. Distribution of modified versions of all or part of this article are permitted, provided that such works carry three things: (1) this copyright notice, (2) prominent notices stating the that it has been changed, and (3) information for how to obtain the original (such as a URL).