 |
version seven. http://demongin.org |
Getting Started with oauth2, identi.ca and Python
Soup to nuts on installing python-twitter and interfacing with the identi.ca API, including authentication via oauth2.
Friday, 2011-03-25 | Careerism, demongin.org, On the Internet, Programming
About a week ago, I decided to phase Twitter out.
And not because of some old hipster bullshit about how it is bogus and unfair to close off API development to developers working outside of the "Twitter ecosystem" (and ban the competition) either: my decision to leave Twitter is based on the company's glaring technical and civic deficiencies.
I actually kind of respect them for explicitly declaring that certain developers, effective immediately, are officially off the reservation.
Ultimately, my decision to drop Twitter came after I sank a not insubstantial amount of time and energy into learning about and understanding open micro-blogging solutions such as StatusNet (formerly Laconica) and free implementations of those solutions, such as identi.ca. Basically, I looked around at how micro-blogging was being done in 2011 by the F/OSS community, and concluded that Twitter, on account of its feature-deficient product1, high rate of (mega-)corporate infiltration, famously frequent capacity problems and unacceptable participation in the American State Department's attempts to persecute Wikileaks supporters, no longer met my needs.
And so, like Microsoft, Trent Reznor and people who send unpunctuated, a-grammatical emails, Twitter is dead to me.
Effective...well, not quite immediately. But damn soon.
Refactor
Since I actually like micro-blogging (and the little embedded status update in the upper-right hand corner of the current layout of http://demongin.org) quite a bit, I am going to have to phase Twitter out, rather than simply dropping it like a hot rock.The first major step of phasing twitter out is choosing a replacement. I settled on identi.ca: it has good client applications (Microblog on Android and Adium everywhere else), high API transparency and all of the features of StatusNet.
The second major step, in my case, was learning how to interact with identi.ca in a programmatic fashion. And the second step, as it usually does, turned out to be a doozy: my old go-to for Twitter automation having been deprecated, I decided to re-standardize on the new, de facto standard Python module for Twitter-like API interfacing, python-twitter. In order to work with the identi.ca API using python-twitter, I rapidly learned that I would have to bite the bullet and crash-course myself in OAuth2, the standard mechanism for authenticating API requests.
What follows, then, is how I got spun up on these two new (to me) pieces of technology: while python-twitter is fairly straightforward and well documented, oauth is easy to screw up and, if you're not in control of the server, can be difficult to troubleshoot.
I've tried to write something halfway between a how-to and a journal. It should be useful for anyone getting started with Python and identi.ca.
Strategy
Even though it's a cool information system once you dig into and start to understand it, if you're coming to oauth cold, it might not exactly make sense (especially if you're in the habit of just embedding a widget or copy-pasting some other dude's javascript).The big idea behind oauth is that you're going to use it to allow users of a service (e.g. identi.ca) to authorize a consumer application (in my case, this blog) to do things on the service for them without their explicit permission. oauth accomplishes this by doing this:
- Instructed by the user, the consumer application contacts the service and requests a token
- The consumer application receives the token and uses it to construct a URL that points to the service
- The user goes to the URL, tells the service that he approves of the consumer application doing things for him without his explicit say-so
- The service application sends a token back to the consumer application, which can then be stored and used whenever the consumer needs to do something on the service on the user's behalf
So, in order to get my programmatic interactions with identi.ca up and running, I'm going to have to first do a little oauth-fu: after that, I'll be able to do basic CRUD work in Django which, though it will not be discussed in this post, is probably fairly easy to imagine.
sysadmin
My first, explortatory RTFM session with python-twitter revealed that, before I could even dig into the development, I had some non-trivial sysadmin work to do: the version of the software that I need to have in order to interact with identi.ca requires a development version that was not in the repository on my server.Ditto the version of oauth that I was going to need in order to run python-twitter.
So, first things first, I wanted to get the existing python-twitter module completely out of the way, in order to save sanity down the line and keep my imports and versions explicit:
root@li66-186:/home/toconnell# aptitude purge python-twitterThen a quick test, just to verify that the module was no longer in the import path of my system-default python interpreter:
toconnell@li66-186:~$ python -c "import twitter" Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named twitterPerfect.
Next thing is to start collecting source tarballs for the development software I was going to be depending upon to make this thing work. Experience teaches that, when doing something stupid (e.g. using software that's not hard-wired to your OS's package management system in a production application), old ways is best ways, so I decided to go with ye olde time sysadmin method of making a sub-dir of /opt called src where I could use a bunch of symlinks to do some primitive release control:
root@li66-186:/home/toconnell# cd /opt/ root@li66-186:/opt# mkdir src root@li66-186:/opt# cd src/Next, start loading up the source files and making the symlinks. The python-twitter module project is hosted on googlecode.com, which sadly has no git support.2oauth2 project3github:
root@li66-186:/opt/src# wget http://python-twitter.googlecode.com/files/python-twitter-0.8.1.tar.gz root@li66-186:/opt/src# tar -zxf python-twitter-0.8.1.tar.gz root@li66-186:/opt/src# ln -s python-twitter-0.8.1/twitter.py root@li66-186:/opt/src# git clone git://github.com/simplegeo/python-oauth2.git root@li66-186:/opt/src# ln -s python-oauth2/oauth2/Now, if you add /opt/src to your sys.path, you should be able to import twitter without any problems:
root@li66-186:/opt/src# cd
root@li66-186:~# python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append("/opt/src")
>>> import twitter
>>> api = twitter.Api(base_url="http://identi.ca/api")
>>> api
<twitter.Api object at 0xb730476c>
OK. Now you're ready to put on your developer hat and use the dev release of python-twitter module and the shiny, next-gen oauth2 module to help automate your interactions with identi.ca.
oauth2
In order to connect to and authenticate against identi.ca, you'll need to gather some information. What follows will walk you through gathering that information in a test/experimental setting: the code below, while fragmentary for instructional/explanatory purposes, could easily be collected and re-written into a single script (or integrated into an existing program).4First, log into the main identi.ca webapp, and pull up the URL where you can manage connections from external applications (currently http://identi.ca/settings/oauthapps). Click the hypertext to register a new oauth application.
Fill in the blanks. Pay special attention to the redirect URL, because you will need to modify your application to capture query strings sent to this URL.
Once you're registered, there's a way to view your "Consumer key" and "Secret key" (as well as token and authorization URL's for identi.ca). Make a note of these: you'll need them to authenticate against identi.ca with the python-twitter module:
At this point, take a quick RTFM break and review the chart in the middle of the oauth.net core doc: the first thing you've got to do is get a request token (i.e. an oauth_token and an oath_secret_token).
Since you're just testing at this point, here is a quick script that will spit out a URL that you can copy/paste into your browser in order to quickly retrieve these things from identi.ca. Adapted from the oauth2 documentation:
#!/usr/bin/env python
import sys
sys.path.append("/opt/src")
import oauth2 as oauth
c_key = "123456678101112131415"
c_secret = "514131211101987654321"
u = "http://demongin.org"
consumer = oauth.Consumer(key=c_key, secret=c_secret)
request_token_url = "http://identi.ca/api/oauth/request_token?oauth_callback=%s" % u
client = oauth.Client(consumer)
resp, content = client.request(request_token_url, "GET")
print resp
print content
Fire that script off, and, with any luck, you'll get a data dump of a dictionary and a string representation of what the identi.ca API returned, which should go a little something like this:
oauth_token=XXXXXXXXXXXXXX&oauth_token_secret=XXXXXXXXXXXX&oauth_callback_confirmed=trueHold on to both the "oauth_token" and the "oauth_token_secret": you'll require the former for an authorization request and the latter for the final access token.
Now that you've got these tokens, the next step is to authorize the "consumer" application (i.e. my blog) access to my user's identi.ca account. To do this, you've got to generate a URL and hit that URL in your browser. To generate the URL, bolt the following bit onto the script you just used to get your "request token" and execute the script:
token = [t.split("=") for t in content.split("&")]
d = {}
for t in token:
k,v = t
d[k] = v
authorize_url = 'http://identi.ca/api/oauth/authorize'
print d
print "%s?oauth_token=%s" % (authorize_url, d['oauth_token'])
You should get a URL that looks more or less like this:
http://identi.ca/api/oauth/authorize?oauth_token=XXXXXXXXXXXXXXXLoad that URL in your browser, and you should be asked whether you (i.e. your identi.ca user) wants to allow your blog access to your account:
Once you approve, you will be directed to the URL you entered when configuring your blog for oauth access to identi.ca. It will look something like this:
http://demongin.org/approved.py?oauth_token=XXXXXXXXXXX&oauth_verifier=XXXXXXXXXXNow that you've got a.) an approved oauth_token and b.) a legit oauth_verifier, you're ready to get the access token that was the goal of this whole project.
Referring back to the handy diagram at oauth.net, you can see that, in order to seal the deal, you've got to sign a new request with these two new inputs. Time for another dummy script:
#!/usr/bin/env python
import urlparse
import sys
sys.path.append("/opt/src")
import oauth2 as oauth
c_key = "123456678101112131415" # The old values you got from identi.ca
c_secret = "514131211101987654321"
consumer = oauth.Consumer(key=c_key, secret=c_secret)
access_token_url = "https://identi.ca/api/oauth/access_token"
o_token = "XXXXXXXXXXXXXXXXXXXXXXXXXX" # The value from the identi.ca redirect
o_token_secret = "XXXXXXXXXXXXXXXXXXXXXXXXX" # The value from the dummy script
o_verifier = "eb3e654b58842806" # also from the identi.ca redirect
token = oauth.Token(o_token, o_token_secret)
token.set_verifier(o_verifier)
client = oauth.Client(consumer, token)
resp, content = client.request(access_token_url, "POST")
access_token = dict(urlparse.parse_qsl(content))
print access_token
Which, finally, should give you something like this:
{'oauth_token_secret': '413121110187', 'oauth_token': '781011121314'}
And that's it: now you're ready to access the "protected resources" of the user who approved your application.
python-twitter
Now, armed with these new hashes and your favorite python interpreter, you should be able to post status updates to identi.ca by following the simple example in the python-twitter documentation:
In [1]: import sys
In [2]: sys.path.append("/opt/src")
In [3]: import twitter
In [4]: api = twitter.Api(base_url="http://identi.ca")
In [5]: api = twitter.Api(base_url="http://identi.ca/api/", consumer_key="123456678101112131415", consumer_secret="514131211101987654321", access_token_key="781011121314", access_token_secret="413121110187")
Make sure you get the URL right when specifying the base_url kwarg: there's lousy error handling here, and if, out of habit, you specify something like "http://api.identi.ca" in stead of the real URL, you'll fail with a (misleading, but not inaccurate) traceback that ends with a JSON parsing problem.
Once you're authenticated and you've got your API object initialized, the sky's the limit:
In [28]: status = api.PostUpdate('I love python-twitter! #test')
- Feature-deficient and, in a lot of cases, just plain stupid. Hash/bangs in URLs? Really? Really?
- Of which, I should mention, I am one.
- Not yet, at least. Rumors are circulating about something in the near future, however.
- You'll need httplib2 for this:
root@li66-186:/opt/src/python-oauth2# sudo aptitude install python-httplib2
- There is a great example of how to do all of this at the googlecode.com URL for python-twitter: I relied heavily upon it while learning/writing this essay.
