How to compare vCard address book entries

Lars Steinke — Fri, 06 Apr 2018 10:46:40 +0000

Following a little mishap with the DAVdroid app on my Fire HDX Android tablet, I ended up with a lot of duplicate contacts in my CardDAV account. Upon closer inspection I then noticed, that in the process of copying the contacts from the local contact store (on my Blackberry smartphone) to the CardDAV account, certain contact details had been lost and spurious characters (like spaces and question marks) introduced.

Additionally, I had started editing the contact details for proper display in the Android contact app by Blackberry: The app does unfortunately not display the organization info provided for contacts not connected to a person (like institutions) in the address book view - contrary to that view on BlackBerry 10 before.

So, I needed an estimate of the mess my contact data was in. An internet search came up with this article, but that method produced way to many hits to be of much use in my case:

The idea then was to give vcardtools found on github a try for normalization, but those crashed and burned (vobject.base.NativeError) upon parsing the VCF data. Ultimately, it came down to nittygritty text manipulation with UNIX console tools, as a preparatory step for further processing:

#!/bin/sh filename=`basename $1 .vcf` cat $1 | grep -v -E '^N(:|;);;;' | sed 's/CHARSET=UTF-8://g' | sed 's/\xe2\x80\x8b//g' | sed 's/?//g' | sed -E 's/^(N|FN|ORG);/\1:/g' | sed 's/ \+/ /g' | sed -E 's/(:|;) /\1/' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | awk 'BEGIN{name="";fname="";org="";prev=""} /^N:/{name=$0}/^FN:/{fname=$0}/^ORG:/{org=$0}/^END:VCARD/{if (name!=""&&fname!="") {print fname;prev=fname} else {if (org!="") {print org;prev=org} else {if (NR>1) print "ERROR"}};name="";fname="";org=""} END{}'| tee "${filename}".cleaned | cut -d":" -f2| sort | tee "${filename}".sorted | uniq -c -i -s 2 | tee "${filename}".counted | cut -c9- > "${filename}".names

First, we eliminate unwanted lines using grep, then we clean out unwanted characters and spaces using sed and finally we select the relevant contact fields (name or organization) usingawkto end up with a manageable data subset, on which to check for differences.

Note: The initial version was based on a comparison with the previous contact entry and while that is not used in the script right now, I have not cleaned it out (as it might prove helpful in the future).

How to repair an ext3 filesystem after mkdosfs format

Lars Steinke — Thu, 05 Feb 2009 20:42:44 +0000

After shrinking a large ext3 partition using gparted to make space for an additional partition, I did not take sufficient care to notice the leading partition I newly created with cfdisk subsequently was denoted as sdc2 instead of sdc1, as one might expect.

Thus woe ensued when I issued a mkdosfs -F 32 /dev/sdc1 command without a closer look at the new partition numbering.

So the interesting question now was, what can be salvaged from an ext3 filesystem with overwritten superblock (as well as a few of the inodes)?Luckily, the erroneous reformatting was FAT32, not ext3, which would have caused the inode block pointers to be zeroed, to quote the ext3 FAQ:
In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas ext2 just marks these blocks as unused in the block bitmaps and marks the inode as "deleted" and leaves the block pointers alone.

What could be done was this:

Find out the location of the backup superblocks, either using
- mke2fs -n /dev/sdc1 or
- testdisk (Advanced>Choose sdc2>Superblock)
Call e2fsck -b 163840 -B 4096 /dev/sdc1 (for a blocksize of 4096 that backup superblock at 163840 should be in the list you produced in the step before)
Check lost+found for what has been be salvaged by e2fsck - not bad at all, without having had to resort to file scrubbing tools like foremost or magicrescue (both available in Ubuntu)
At actual loss were the inodes directly overwritten by the two FATs - sort of a black eye I came away with, really...

lsweb.eu

How to compare vCard address book entries

How to repair an ext3 filesystem after mkdosfs format