Newer Post » « Older Post

Import your del.icio.us bookmarks into Google #

Date: Monday, October 16, 2006
Labels:

I've written a simple script/site that lets you import your del.icio.us bookmarks into Google's bookmarks feature (which doesn't seem to have a product name - oh, right):

http://persistent.info/delicious2google/

Technical Details

For a while now, Kushal* had been wanting a way to import his del.icio.us bookmarks into Google. There are some advantages to this, including the fact that your tags labels show up in search results if they contained a bookmarked site and one-click Toolbar access. It was pointed out to me that the bookmarks site has a POST/XML-based bulk import interface that the Google Toolbar uses to import IE Favorites. I initially thought that I could use del.icio.us's JSON API to get the list of bookmarks, thus having a completely client-side solution that did not require the user to hand over his/her credentials. Unfortunately that turned out to be limited to 100 posts. Hack-ish workarounds came to mind, like getting the list of tags first and then getting the bookmarks for a particular tag (still wouldn't work if there were more than 100 bookmarks for a single tag). I abandoned that approach, but for the sake of posterity, that version is archived here.

Since del.icio.us also has a REST API that allows full access to data, my next thought was to use that. pydelicious is a pretty simple to use Python library for accessing the API, and it was easy to create a script that used it to fetch the list of bookmarks for a user. Since I already had the code from my previous attempt that took the JSON output and uploaded it, I merely made the Python code output similar JSON and used the same upload method.

This approach seems to work pretty well, but it has the caveat that the user has to enter their username and password into a third-party site. I've provided the source code so that the paranoid can run their own copy. While the Google bookmarks site doesn't have an API per se, it does export its data as RSS feeds thus this does not have to be a one-way transfer; an enterprising hacker could write an importer in the other direction.

* His newly-released luxur(n)y site is interesting if you're in the market for a NYC luxury rental.

49 Comments

You rock! How about integrating Google Reader's Shared Items with Google Bookmarks next? :-) That's the only thing that stands between turning Google Reader/Bookmarks into a del.icio.us replacement for me. -ian
Posted by: Ian Murdock February 10, 2007 11:01 PM
Hmmm...looks like it only imported 36 of my bookmarks.
Posted by: DK February 10, 2007 11:01 PM
Is there any particular reason this couldn't be implemented for Digg bookmarks? I'm not trying to be lazy, I'm wondering if you've checked it out before I attempt to do it myself.
Posted by: Nelson Blaha February 10, 2007 11:01 PM
Any chance of getting this to import descriptions? Also back dating of bookmarks to the dates they were saved in Del.icio.us?

If only Google Bookmarks were sharable...
Posted by: Erasable Ink February 10, 2007 11:01 PM
I really wished Google Bookmarks would be fleshed out into a full delicious style site. Right now it's very limited. Heres hoping its the next Google product that gets some real attention.
Posted by: badrad February 10, 2007 11:01 PM
Works nicely. Now if these bookmarks would be used for search personalization.. Maybe once you guys have enough CPU to calculate a page rank per user :)
Posted by: Gregor J. Rothfuss February 10, 2007 11:01 PM
if you feel unsafe about revealing the password, you can just change it for the time the import is running and revert it back to the original afterwards.

Nice script!! Though I have 630 items in delicious and only about 70 were imported ... any idea what to do?

thanks!
Posted by: k February 10, 2007 11:01 PM
I was able to import quite a few bookmarks, but not all. I will keep an eye here to see if there are any updates.
Posted by: thomashodder.com February 10, 2007 11:01 PM
Great script!
Posted by: namespacesolutions.com February 10, 2007 11:01 PM
I completely agree with the first post by Ian. Google reader must be integrated with Google Bookmarks, particularly the shared items. Also, there should be a bookmarklet which lets you tag and share the article AFTER you have read it. It doesn't make sense to share a feed when all you've read is the headline and a paragraph.

Jonathan
Posted by: Jonathan February 10, 2007 11:01 PM
Jonathan: Agreed. And if Bookmarks and Reader were integrated, you could just use the existing Bookmarks bookmarklet to do that. -ian
Posted by: Ian Murdock February 10, 2007 11:02 PM
Another thing, I'm in love with your Smart Google Reader Subscribe userscript. Perhaps a similar script for Google Bookmarks. Anyway, thanks.
Posted by: Erasable Ink February 10, 2007 11:02 PM
Reader and Bookmarks would be great together!

I've been using Reader regularily since it started, and I use stars for quick scanning (a "to read" list to return to when I don't have time to do it). However, I also keep my interesting links there, for tranferring to del.icio.us. If I was using Bookmarks and the feed items had an easy link to add to it, life would be great.
Posted by: Peter Organisciak February 10, 2007 11:02 PM
Oh. I had no clue about Google Bookmark's XML importing interface. Thanks so much for the info, Mihai!

Btw, I threw this into an almost one-step bookmarklet that keeps bookmark and login data client-side. With it, one need only visit delicious' export page and click the bookmarklet to sync everything.

http://blog.lifeslip.com/articles/2006/10/25/bookmarklet-delicious-google-bookmarks
Posted by: Kevin Ko February 10, 2007 11:02 PM
I would really like to import del.icio.us bookmarks into Google Bookmarks and keep my notes/descriptions intact.

I don't know enough about Python scripting to edit your script and run it myself, but I played around a bit with the del.icio.us and Google Bookmarks APIs in order to figure out how they work.

If you map the contents of the attribute "description" as output from the del.icio.us REST API into the Google Bookmarks attribute "annotation," the notes should be transferred. ("&annotation=" is the string that will add notes when part of a bookmarklet; I'm using "&annotation='+getSelection()" to set the notes field to the current selection in the bookmarklet I'm currently working on).
Posted by: Brad February 10, 2007 11:02 PM
By the way, if you do update your import script, it would be really cool if you could find a way to transfer the "time" del.icio.us attribute so that the date bookmarks were added could be moved over as well. I'm not sure that it would have a corollary in the Google Bookmarks import interface, though.
Posted by: Brad February 10, 2007 11:02 PM
Argh... sorry for the triple-post. I just looked at the del.icio.us API again, and the "extended" attribute is the one that should end up as notes in Google Bookmarks. "description" is just the link title, as you no doubt already know.
Posted by: Brad February 10, 2007 11:02 PM
Brad, unfortunately it looks like the Google Bookmarks import interface does not pass through the description.
Posted by: mihai February 10, 2007 11:02 PM
Anyone know why the RSS feed for each label only calls up the first 15 bookmarks?
Posted by: Eric February 10, 2007 11:02 PM
I'm developing an application for importing and exporting Google bookmarks from text strings and find this article very helpful in getting the Google interface. I don't see the Google interface in your latest python code, but in your "abandoned" approach I was able to see the XML tags used in sending bookmarks to Google. Your "abandoned" approach results in another Google window which asks for confirmation ("do you really want to import these"). By pusing the "Import" button and using tcpflow I was able to find out the interface which will "just import" the bookmarks in one step; you can skip the confirmation. This shortcut is that you send an HTTP POST to:

"http://www.google.com/bookmarks/mark?op=upload_selection&hl=en&zx="

The body of the post begins with:

q=&s==Import

and then you follow this by any number of bookmark data streams, like this one:

&bm_1=X&url_1=&ttl_1=&lbl_1=

The next one is bm_2, url_2, ttl_2, lbl_2, etc.

This all seems to work fine, except that I cannot figure out the tag used to import the fourth and final field, the "Notes" or "Annotation" for each bookmark. You don't have that in your code. I've tried ano_1, ann_1, not_1, etc. No luck!

In the RSS feed of bookmarks that comes back from Google, I see that they use the longer name "annotation", along with the "label" and "title", but I tried "annotation", "note" and "annotations" in your "abandoned" code but these don't work either.

Do you know what tag might be used for "notes" or "annotations"??

By the way, parsing the RSS to read the bookmarks is easy, like you said. I've also figured out how to delete bookmarks. So this is the last lilttle piece I need to have a complete Google Bookmarks importer/exporter!

Jerry Krinock
San Jose, CA USA
Posted by: Jerry Krinock February 10, 2007 11:02 PM
Argghhh! I just noticed that blog ate up some of the symbols in those two lines of code my previous post because it thought they were html tags. Here's what they should be:

q=&s=signatureFromRSSHeaderItems=Import

&bm_1=X&url_1=urlOfBookmark1&ttl_1=titleOfBookmark1&lbl_1=commaSeparatedLabelsOfBookmark1

Jerry
Posted by: Jerry Krinock February 10, 2007 11:02 PM
After reading the comments above by Kevin, Brad and doing further investigation, I agree with Mihai that there is no tag in the "Import Bookmarks" interface supporting import of "Notes". There is an obvious reason ro this: It appears that this "Import Bookmarks" interface which Mihai has exploited was designed by Google for importing "Internet Explorer" bookmarks. Since Internet Explorer does not support "Notes", there is no reason to support importing of them. Argghhhh!!
Posted by: Anonymous February 10, 2007 11:02 PM
By the way, the previous post was mine. Forgot to enter my name. Regarding Eric's question, I'd say the answer is because 15 is the number of bookmarks you get on a page when you load google.com/bookmarks. So, now we need to figure out how to get the rest of them....
Posted by: Jerry Krinock February 10, 2007 11:02 PM
Ah, that was easy! Simply append the query item start=15, start=30, etc. to successive requests. To see how it works, if you have more than 15 Google bookmarks, enter this in to your browser's address field:

http://www.google.com/bookmarks/lookup?start=15&output=rss
Posted by: Jerry Krinock February 10, 2007 11:02 PM
Worked like a charm. Looks like all of my bookmarks transferred over. Thank you!!!
Posted by: Hisao Yatsuhashi February 10, 2007 11:02 PM
I just discovered one other thing about that RSS feed of bookmarks you get from Google (when "exporting from Google"). The header says

encoding="utf-8"

but the accented "e" (as in "San Jose") comes to you as a single byte, 0xE9. That aint' UTF-8!!! So, I have to hack into it and replace that with

encoding="windows-1252"

and now my XML parser is much happier.

Since Google Bookmarks is not a public API, there is no place to discuss it on Google; this seems to be the only place where people are interested!
Posted by: Jerry Krinock February 10, 2007 11:02 PM
Imported 1151 bookmarks. sweet.
Posted by: iblee February 11, 2007 11:32 PM
Jerry, thanks for noticing that encoding bug. It should be fixed now, according to the Search History team.
Posted by: Mihai Parparita February 15, 2007 2:13 PM
thanks for the script/service... it rocks.
Posted by: Anonymous April 1, 2007 2:45 PM
Thanks for the script!
Posted by: puzza007 May 1, 2007 6:06 PM
Hi,

This is about the third time I am trying to upload my delicious to Google Bookmarks. But it just displays the "Uploading" page (after I give the delicious userid and pwd) and nothing much seems to happen after that.

My Google Bookmark count still remains at 4 (those 4 are the manually added ones).

Any idea what could be the reason?

I have close to 1000 items in delicious. Also, it would be very helpful if it can display some status message like "uploading 1 of 1000" etc. At least, the people who are uploading would know what is happening.

Thanks so much for the great blog and all the great greasemonkey scripts that you have written over the last few years.

Regards,
Proto, India.
Posted by: Proto May 3, 2007 3:18 AM
I've written a script which will do exactly the opposite - ie move bookmarks from Google to del.icio.us, that is. :-)

It's a command-line Python program which can be run from your desktop. You can grab it here - http://code.google.com/p/gbookmark2delicious/
Posted by: Anoop Johnson May 6, 2007 11:53 PM
Having the same problem Proto describes above. Enter username and password and the page just reads "Uploading..." and nothing further happens.

Tried both Firefox and IE.
Posted by: Kamholio May 17, 2007 1:03 AM
having the same problem as Proto.
hope it coгдв be fixed.

Alex
Posted by: Alex Riabtsev May 17, 2007 7:31 AM
Me three. (Actually, it imported a single del.icio.us bookmark for me, of 1000's, and then subsequent attempts result in the hung "Uploading..." page.)

I'm not complaining -- this is free code, after all, and I'm not jumping up to debug it. But FYI.
Posted by: Anonymous June 8, 2007 2:09 AM
Thank you very much! I just imported my bookmarks (1000+), and (I think) they were all copied properly. I love it!

If it would be possible to have Google reader be able to interact with your favorites so you would have the easy option of adding the feeds from one section to the other. (^_^)
Posted by: Sir Onizuka July 13, 2007 2:45 PM
Wow, this is too cool man. You Rock!
Posted by: Anonymous August 13, 2007 1:16 PM
wow this is easy and works fast I ma very impressed
Posted by: Hoodia Gordonii Plus Weight Loss August 20, 2007 1:46 PM
Me four...I see the Uploading Page and the Progress Bar on my Browser increases, but when it's finished it takes me to the Gmarks Import Page but no Bookmarks are listed there.
Posted by: moulari November 13, 2007 6:43 PM
Is there a way to Synchronize your Bookmarks in del.icio.us Google Bookmarks and your Browser including the date of publishing, associated tags notes and frequency of visit.

del.icio.us lets you share your Bookmarks but doesn't have the capability that Google Notebooks integration with Google Bookmarks offer.

With Firefox 3 supporting tags it will be really useful.

I am currently looking for ways to at least synchronize the Bookmarks including the associated tags between Google Bookmarks and del.icio.us

I have both the del.icio.us adon and all the Google adons for Firefox (Google Browser Sync, Google Notebook Ad-on, Google Toolbar which includes Bookmarks but can't find a way to synchronize the Bookmarks.)

The reason I am looking at it is that I realized just now that all your Google Bookmarks are already accessible from your Google Notebook. It would be amazing to have notes for everything you find on the Web..

Your script helped a lot..

You may consider an ad-on for Firefox3 as they are implementing tags so I guess a lot of interoperability issue will be resolved.

Thanks for the Script...

And the best part was that you managed to get this done without taking the Google Account info.

In the first page where the Script asks for your del.icio.us login info I was a little worried thinking that I might need to provide my Google Account information but then the Transparency that you demonstrated by providing the full source code of what will get executed in teh Server did give me a lot of confidence in your Script.

Also needless to mention no redundancy of bookmarks a s it filters already existing bookmarks, with the option to choose which bookmarks you want to import

Thanks man..
Posted by: Bijay Rungta November 16, 2007 3:36 PM
You can also consider making a few tweaks to merge the new Tags being added to ignore case sensitivity and singular/plural things which I think is one of the biggest Problem with folksonomy.

Also the fact that Google allows you to have multi word tags whereas del.icio.us doesn't is again an interoperability issue.

Bye the Way I read the last para of the post after importing my bookmarks and posting my last comment. SO you can probably understand my concerns on the Login info issue. :)
Posted by: Bijay Rungta November 16, 2007 3:48 PM
Fantastic! That worked like a charm with no problems at all. Great job.
Posted by: Brian December 21, 2007 8:27 PM
Simple way to search on sites saved on del.icio.us - use deliGoo (www.deliGoo.com). deliGoo can create custom search engines by your demand from any del.icio.us user's bookmarks and/or tag.
Posted by: DM January 6, 2008 11:01 AM
It is really annoying to not be able to upload the annotations / notes. I can't believe Google left this out of their op=upload code.
Posted by: Anonymous February 7, 2008 10:36 AM
Is it possible to add a feature so if bookmark already exist then your scripts add labels available in Delicious?

Thanks!
Posted by: Shital March 16, 2008 4:25 AM
am i able to do the same with simpy.com? simpy does also use REST API
Posted by: Viktor April 10, 2008 9:08 AM
for some reason, when i enter my username and password it shows the import bookmarks page with the list of bookmarks to import and then it refereshed automatically and the list of bookmarks that appeared before doesn't anymore
Posted by: Pratik Sinha June 1, 2008 1:54 AM
I'm having the same problem as pratik....
Posted by: Daniel June 27, 2008 1:08 PM
works fine until upload repeatedly stops after importing 568 bookmarks
Posted by: Anonymous September 9, 2008 1:56 PM

Post a Comment

Newer Post » « Older Post
Linkblog