Monday, 25 April 2016

Query your firefox browsing history with sqlite


I was search for a number that I had rang from an advertisement I seen on the internet. A few months later I needed that number. I went to the url and found the ad had expired. There was no number available. I have an itemized phone bill but the number could be any where with in the last 6 months. I needed to narrow down the search.

Information that I knew:
  • The url of the advertisement
  • I rang the number around lunchtime.

Firefox history

I checked my firefox browser history. I thought that if i knew the date I visited the advertisement it would reduce the number of telephone numbers I had to choose from. The browser history page was not detailed enough for that.

2 issues with the firefox history browser
  • It organizes the website you visited into what month you visited it if it older than a month. There is no column for exact date or time of the visit.
  • When using the searchbox the url would appear in the results but it would not tell you what month,date or time you visited the url.

Firefox history database

In your home directory firefox keeps all the history in a sqlite file.
~/.mozilla/firefox/<random directory>/places.sqlite
To find your database run this command
find ~/.mozilla/firefox/ -name places.sqlite
In the moz_places table it has an epoch in microseconds of the exact time you last visited the website. Unfortunately if you visited the site more recently you only have that date. Firefox does not keep a record of every time you visited a url. By chance I visited this url on 2 different computers so one computer still had my first visit time while the other had the time that I visited the expired advertisement.

I was then able to get the exact url and query the table. I converted from microseconds to a date in one query.
select datetime((select last_visit_date from moz_places where url='<EXACT URL>')/1000000,'unixepoch', 'localtime');

I found the date I visited the url. I was able to see on my phone bill the next day that I rang a number I did not know. I rang the number and it turned out to be the person I was looking for. The query helped me narrow down my search.

Thursday, 14 April 2016

Java nio move under the hood

Moving files with java.nio it will figure out if you are moving a file on the same partition or to a different partition.
  • If you are moving on the same file system then it will do a move which is faster than a copy.
  • If you moving to another filesystem then it will do a copy.
Do not always assume move is fast in java nio. If there is issue with the move it will can mask the issue and do a copy of the file instead.
By looking through the code below you can see.

This is the move method in the Files class. It decides if should do a move or
copy based on if the file is moving with in the same file system or to another  one.
  public static Path move(Path source, Path target, CopyOption... options)  
     throws IOException  
     FileSystemProvider provider = provider(source);  
     if (provider(target) == provider) {  
       // same provider  
       provider.move(source, target, options);  
     } else {  
       // different providers  
       CopyMoveHelper.moveToForeignTarget(source, target, options);  
     return target;  

FileSystemProvider is an abstract class that provides methods for concrete classes to provide a move method that is associated with the file system. 
You will have a different move implementations for every file system.

For linux filesystems this class provides the move method that just calls UnixCopyFile.move()
In this class it does a rename on the files from source to target. If it can not complete that job then it will just do a copy which will transfer the bytes. Look at the code below. The exception can be ignored.
If the rename exception is not bad enough it will ignore and continue with a copy.
 // first try rename  
     try {  
       rename(source, target);  ######  
     } catch (UnixException x) {  
       if (x.errno() != EXDEV && x.errno() != EISDIR) {  
         x.rethrowAsIOException(source, target);  
     // copy source to target  
     if (sourceAttrs.isDirectory()) {  
       copyDirectory(source, sourceAttrs, target, flags);  
     } else {  
       if (sourceAttrs.isSymbolicLink()) {  
         copyLink(source, sourceAttrs, target, flags);  
       } else {  
         if (sourceAttrs.isDevice()) {  
           copySpecial(source, sourceAttrs, target, flags);  
         } else {  
           copyFile(source, sourceAttrs, target, flags, 0L);  ######  
This C file has the native JNI rename method for linux

In the end you could be faster to do a rename instead of a move if you know your application is working on the same file system. Less decisions will be made by java for you. 

Sunday, 10 April 2016

Sqlite 2 databases can not be correctly read by sqlite3

If you try to open a sqlite file that is created with sqlite 2 with sqlite3 command then you can get an error like this when running commands on that database

Error: file is encrypted or is not a database

The database file must be created and read by the same version of sqlite. Sqlite3 will not be able to see the tables created in the sqlite2 database.
This can also cause a problem if you are using the sqlite 3 jdbc connector with a sqlite database file that was created with sqlite2. Queries on this file will say that the table you are trying to access does not exist.

Tuesday, 13 May 2014

Using openhab bluetooth binding on linux

OpenHab does not distribute the bluecove GPL library in the Bluetooth binding because it is GPL licensed. OpenHAB is EPL (Eclipse Platform License) which is friendly to business since they do not have to open source their own bindings.
You will get this error if you do not modify the Bluetooth binding:
To fix the problem follow these steps:
unjar the org.openhab.binding.bluetooth-1.4.0.jar bundle
change directory to lib

Download the blue cove GPL jar from here

Add the Bluecove GPL library in lib directory of the bundle along with the blue cove jar

Add the new library to manifest:
Bundle-ClassPath: .,lib/bluecove-2.1.1-SNAPSHOT.jar,lib/bluecove-gpl-2.1.0.jar
create the jar file with the original manifest file
jar -cvmf META-INF\MANIFEST.MF org.openhab.binding.bluetooth-1.4.0.jar *

This will solve your bluetooth problem.

Tuesday, 8 April 2014

Having a keystore with 2 certificates

You are able to have more than 1 key and certificate chain in your keystore. A problem is only when you are using the default key manager to pick the cert to represent your client to the server. If the wrong cert is picked for representation then the server will no accept you a trust client.

In this example you have 2 aliases. Both are key certificate chain entries in the keystore. Both aliases have the same CN (common name) in their certs. The default key manager will find the matching aliases with the same same CN from what I can see from my testing. If you have 2 certs that are using the same CN then you might have problem since you do not know what cert the default key manager will pick to represent the client to the server.

Snippet output from
You will see this print out after the server has requested the client certificate.
*** ServerHelloDone
matching alias: client1
matching alias: client2
*** Certificate chain
It picks the first in the list of its print out.
The rest of the print out will the be the details of the client1 alias in the keystore which it uses to identify the client to the server. It imght of picked the wrong alias which will give you a unknown_ca error.

What I noticed about the default keymanager picking aliases

  • does not pick them alphabetically.
  • does pick the first entry in the keystore

Only place i could find this information is here.

If you need to use certificates that have very similar attributes then you have to implement you own custom KeyManager so you can choose the correct alias to represent your client to the server. If only it was easy to configure the default KeyManager....

Saturday, 25 January 2014

SCCS cheat sheet

Source Code Control System is on old version control systems. It predates RCS which predates SVN. You can see how old it is now. Some companies still use in their legacy systems. To remind myself I have made a cheat sheet.
All commands start with a call to the sccs binary. For example "sccs edit".

Check out a new version of a file in SCCS

Just like uncheckout

Will get the latest file from the SCCS directory to your current location

Add a file to SCCS

This will remove all files in the current directory that you are able to retrieve again from SCCS. This is better then delete the file using a linux or unix command because you might delete files that are not under SCCS control.

check : Used in scripts. It returns a non zero exit status if a file is being edited. It will check what files are being edit in the current directory. For this command you have to be in the directory where the SCCS directory is. IT also provides more information than tell e.g user,version,date it was checked out

tell : Used on the command line to inform user if files are being edit in the current directory. It checks the SCCS directory for the information.

Will show the difference between the current version and previous version you specified.

Thursday, 5 May 2011

Converting MP3 to M4B audio books for an ipod in linux

I recently bought an ipod for listening to audiobooks. The reason for the ipod is that it has an audiobook feature. The ipod has a special file format for audio books called m4b If the audio book is in a m4b format the ipod will track where in the audio book you press paused. Then you can play other music. When you come back to play the audio book it will remember your remember playback position. Mp3 does not have this feature. If the audio book is in the file format mp3 then you will have to remember where you stopped listening before you can play other music. Then when you play the mp3 audio book again you have to forward to that time where you left it last. Also if the file is in m4b it will show up in the ipod audiobook section. Converting 1 file This will turn one MP3 in ipod M4B. This will default to converting the mp3 to a M4B file at a sample rate of 44000Hz. -b 80 below is the bit rate you want the m4b file to be.

mpg123 -s "Memory techniques.mp3" | faac -b 80 -P -X -w -o "Memory techniques.m4b" -

Changing the sample rate of the file
If the audio book out put is slow compared to the mp3 when you hear the m4b file. For instance the person reading the audio book is talking slow. The problem is you are converting a mp3 that does not have a sample rate of 44000Hz to a m4b file with a sample rate of 44000hz. Find the sample rate of the mp3 file and add it as a input in the command. For instance the mp3 below has a sample rate of 24000Hz. Now the M4B file will have a sample rate of 24000Hz. The audio output of the M4B file will be the same as the mp3.

mpg123 -s "Even Better Memory techniques.mp3" | faac -b 80 -R 24000 -X -w -o "Even Better Memory techniques.m4b" -
Also if the file playback is talking too fast then you need to decrease the sample rate. You could of made the file with a sample rate of 44KHz and person is talking fast. Decrease to 24Khz and see if the person speach has slowed down to a normal level.
Converting multiple mp3s to one m4b file
Alot of the time an audio book is broken into multiple mp3 files. Instead of having multiple m4b files you can convert all the mp3 files in one m4b file. This command will get all the mp3 files in a directory and add them all into one m4b file.
First get all the mp3 names in to one file
ls *.mp3 > inputfile
Then start this command to have all the mp3 in one m4b file to listen to on your ipod and have the bookmark feature.
mpg123 -@ inputfile -q -w - | faac -w  --track "1/1"  --artist "<AUTHOR>" --title "<TITLE>" --album "<TITLE>" --writer "<AUTHOR>"  --genre "Audiobook" -o "DISC1.m4b" -