Saturday, 4 June 2016

Building Eclipse from Scratch

I read a mailing list that outlines the reason why Debian unstable still has a Eclipse version that is 4 years old. No one has been able to package it up fully and maintain it since version 3.8 of Eclipse. The latest version of eclipse is 4.5 (Mars)

Then I thought that I have never built Eclipse from scratch previously. I wondered how hard it would be.

Thankfully they have made it easy by using a git repo that has a list of sub modules. Everything you need is in the platform aggregator repo.

By follow Section 13.4 in this tutorial you can build Eclipse 4.x. Previously the build was done differently. The new way to build is with Tycho.

Run this command and you will see the GIT project is made up of all other projects that together form the Eclipse Java Develop Tool IDE.

 ~/eclipsebuild/eclipse.platform.releng.aggregator$ git config --file .gitmodules --name-only --get-regexp path  

Monday, 25 April 2016

Query your firefox browsing history with sqlite


I was search for a number that I had rang from an advertisement I seen on the internet. A few months later I needed that number. I went to the url and found the ad had expired. There was no number available. I have an itemized phone bill but the number could be any where with in the last 6 months. I needed to narrow down the search.

Information that I knew:
  • The url of the advertisement
  • I rang the number around lunchtime.

Firefox history

I checked my firefox browser history. I thought that if i knew the date I visited the advertisement it would reduce the number of telephone numbers I had to choose from. The browser history page was not detailed enough for that.

2 issues with the firefox history browser
  • It organizes the website you visited into what month you visited it if it older than a month. There is no column for exact date or time of the visit.
  • When using the searchbox the url would appear in the results but it would not tell you what month,date or time you visited the url.

Firefox history database

In your home directory firefox keeps all the history in a sqlite file.
~/.mozilla/firefox/<random directory>/places.sqlite
To find your database run this command
find ~/.mozilla/firefox/ -name places.sqlite
In the moz_places table it has an epoch in microseconds of the exact time you last visited the website. Unfortunately if you visited the site more recently you only have that date. Firefox does not keep a record of every time you visited a url. By chance I visited this url on 2 different computers so one computer still had my first visit time while the other had the time that I visited the expired advertisement.

I was then able to get the exact url and query the table. I converted from microseconds to a date in one query.
select datetime((select last_visit_date from moz_places where url='<EXACT URL>')/1000000,'unixepoch', 'localtime');

I found the date I visited the url. I was able to see on my phone bill the next day that I rang a number I did not know. I rang the number and it turned out to be the person I was looking for. The query helped me narrow down my search.

Thursday, 14 April 2016

Java nio move under the hood

Moving files with java.nio it will figure out if you are moving a file on the same partition or to a different partition.
  • If you are moving on the same file system then it will do a move which is faster than a copy.
  • If you moving to another filesystem then it will do a copy.
Do not always assume move is fast in java nio. If there is issue with the move it will can mask the issue and do a copy of the file instead.
By looking through the code below you can see.

This is the move method in the Files class. It decides if should do a move or
copy based on if the file is moving with in the same file system or to another  one.
  public static Path move(Path source, Path target, CopyOption... options)  
     throws IOException  
     FileSystemProvider provider = provider(source);  
     if (provider(target) == provider) {  
       // same provider  
       provider.move(source, target, options);  
     } else {  
       // different providers  
       CopyMoveHelper.moveToForeignTarget(source, target, options);  
     return target;  

FileSystemProvider is an abstract class that provides methods for concrete classes to provide a move method that is associated with the file system. 
You will have a different move implementations for every file system.

For linux filesystems this class provides the move method that just calls UnixCopyFile.move()
In this class it does a rename on the files from source to target. If it can not complete that job then it will just do a copy which will transfer the bytes. Look at the code below. The exception can be ignored.
If the rename exception is not bad enough it will ignore and continue with a copy.
 // first try rename  
     try {  
       rename(source, target);  ######  
     } catch (UnixException x) {  
       if (x.errno() != EXDEV && x.errno() != EISDIR) {  
         x.rethrowAsIOException(source, target);  
     // copy source to target  
     if (sourceAttrs.isDirectory()) {  
       copyDirectory(source, sourceAttrs, target, flags);  
     } else {  
       if (sourceAttrs.isSymbolicLink()) {  
         copyLink(source, sourceAttrs, target, flags);  
       } else {  
         if (sourceAttrs.isDevice()) {  
           copySpecial(source, sourceAttrs, target, flags);  
         } else {  
           copyFile(source, sourceAttrs, target, flags, 0L);  ######  
This C file has the native JNI rename method for linux

In the end you could be faster to do a rename instead of a move if you know your application is working on the same file system. Less decisions will be made by java for you. 

Sunday, 10 April 2016

Sqlite 2 databases can not be correctly read by sqlite3

If you try to open a sqlite file that is created with sqlite 2 with sqlite3 command then you can get an error like this when running commands on that database

Error: file is encrypted or is not a database

The database file must be created and read by the same version of sqlite. Sqlite3 will not be able to see the tables created in the sqlite2 database.
This can also cause a problem if you are using the sqlite 3 jdbc connector with a sqlite database file that was created with sqlite2. Queries on this file will say that the table you are trying to access does not exist.

Tuesday, 13 May 2014

Using openhab bluetooth binding on linux

OpenHab does not distribute the bluecove GPL library in the Bluetooth binding because it is GPL licensed. OpenHAB is EPL (Eclipse Platform License) which is friendly to business since they do not have to open source their own bindings.
You will get this error if you do not modify the Bluetooth binding:
To fix the problem follow these steps:
unjar the org.openhab.binding.bluetooth-1.4.0.jar bundle
change directory to lib

Download the blue cove GPL jar from here

Add the Bluecove GPL library in lib directory of the bundle along with the blue cove jar

Add the new library to manifest:
Bundle-ClassPath: .,lib/bluecove-2.1.1-SNAPSHOT.jar,lib/bluecove-gpl-2.1.0.jar
create the jar file with the original manifest file
jar -cvmf META-INF\MANIFEST.MF org.openhab.binding.bluetooth-1.4.0.jar *

This will solve your bluetooth problem.

Tuesday, 8 April 2014

Having a keystore with 2 certificates

You are able to have more than 1 key and certificate chain in your keystore. A problem is only when you are using the default key manager to pick the cert to represent your client to the server. If the wrong cert is picked for representation then the server will no accept you a trust client.

In this example you have 2 aliases. Both are key certificate chain entries in the keystore. Both aliases have the same CN (common name) in their certs. The default key manager will find the matching aliases with the same same CN from what I can see from my testing. If you have 2 certs that are using the same CN then you might have problem since you do not know what cert the default key manager will pick to represent the client to the server.

Snippet output from
You will see this print out after the server has requested the client certificate.
*** ServerHelloDone
matching alias: client1
matching alias: client2
*** Certificate chain
It picks the first in the list of its print out.
The rest of the print out will the be the details of the client1 alias in the keystore which it uses to identify the client to the server. It imght of picked the wrong alias which will give you a unknown_ca error.

What I noticed about the default keymanager picking aliases

  • does not pick them alphabetically.
  • does pick the first entry in the keystore

Only place i could find this information is here.

If you need to use certificates that have very similar attributes then you have to implement you own custom KeyManager so you can choose the correct alias to represent your client to the server. If only it was easy to configure the default KeyManager....

Saturday, 25 January 2014

SCCS cheat sheet

Source Code Control System is on old version control systems. It predates RCS which predates SVN. You can see how old it is now. Some companies still use in their legacy systems. To remind myself I have made a cheat sheet.
All commands start with a call to the sccs binary. For example "sccs edit".

Check out a new version of a file in SCCS

Just like uncheckout

Will get the latest file from the SCCS directory to your current location

Add a file to SCCS

This will remove all files in the current directory that you are able to retrieve again from SCCS. This is better then delete the file using a linux or unix command because you might delete files that are not under SCCS control.

check : Used in scripts. It returns a non zero exit status if a file is being edited. It will check what files are being edit in the current directory. For this command you have to be in the directory where the SCCS directory is. IT also provides more information than tell e.g user,version,date it was checked out

tell : Used on the command line to inform user if files are being edit in the current directory. It checks the SCCS directory for the information.

Will show the difference between the current version and previous version you specified.