Download your Twitter posts in Terminal

If you’d like to backup your Twitter posts, you can do that fairly easily, with a few caveats:

1. You have to install twurl and setup OAuth. This is not difficult (assuming that you’re the kind of command line geek who is likely to read these words) but it will take 10-15 minutes to do the first time. Once you have it installed, you can use twurl for a whole bunch of things on the command line. It’s not as easy as curl used to be, but oh well.

2. Twitter will only give you access to your most recent 3200 (or so) posts. If you have more than that, they still exist, but there’s no easy way to get them and Twitter won’t help you.

3. This script uses the ‘page’ functionality, which I think Twitter said they’re going to do away with, in their never-ending quest to make things more difficult.

Here’s the script.

#!/bin/sh

# Your twitter name, without the @
TWIT=YourTwitterName

# we're going to loop through this starting at 1. 
COUNT=1

# loop through this as long as we haven't exceeded 17 times
# since we're asking for 200 per page and they give us 3200
# 16 is the most we can get. You can ask for more
# but it probably won't work
while [ "$COUNT" -lt "17" ]
do
    # save each page's worth of output in a different file
    FILE="$TWIT.backup.$COUNT.xml"

    # use twurl to get the infomation
    (twurl -t "/1/statuses/user_timeline.xml?screen_name=$TWIT&count=200&page=$COUNT" 2>&1) |\
     cat -v > "$FILE"

    # the '-t' flag gives us access to status replies. We want 'status 200'
    # if we didn't get it, don't increment the counter, do it again
    egrep -q '^\-> "Status: 200 OK\\r\\n"' "$FILE"

    # check to see if egrep exited properly
    EXIT="$?"

    if [ "$EXIT" = "0" ]
    then

        # if it exited properly, increment the counter
        echo "$NAME: Done with $COUNT"
        COUNT=`expr $COUNT + 1` 

    else

        # if it didn't, try again
        echo "$NAME: $COUNT failed, trying again"

    fi

done
exit 0

That’s it.

Of course that’s going to give you the raw XML which isn’t very pretty to look at. You can get the text and the URL of your posts quickly by using this. (Note: replace “TWITNAME” below with your Twittername.)

egrep -h -A1 '^  <id>' *.xml |\
sed 's#.*<id>#http://twitter.com/TWITNAME/status/#g; s#</id>##g; s#.*<text>##g; s#</text>##g'

That’s just one way, of course. The rest, as my Computer Science professor used to say, is left as an exercise for the reader.

1. cpillsbury liked this
2. toldorknown liked this
3. dwineman liked this
4. cleverhacks reblogged this from luoma
5. darkuncle liked this
6. cocktailstraw liked this
7. morrowplanet liked this
8. luoma posted this