Monday, September 8, 2008

What is SaleRadar?

So what is SaleRadar?  The idea came from driving around and seeing sparsely-attended garage sales throughout my home town, and I thought I might be able to build an easy-to-use system for advertising these sales, that might encourage a sort of grassroots economy to help people in the current downturn in the economy.

The design philosophy I wanted to follow is what I call the three S's: Simple, Simple, Simple.  I am planning to add more features later, but at the core of it, I wanted people to be able to choose if they wanted to post or search, and to enter and address where they were either hosting or searching for garage sales.

I got down and dirty with the Google Maps API, and after a little wrangling, I got some basic functionality.  Building on one of their samples, I was able to do some searching and posting.  One word to the wise: pay attention to that clap-trap about not "unrolling" the marker object.  If you want to show multiple markers, you have to take an extra step that's a little counter-intuitive, but that's well documented (although I ignored it at first), so I won't cover it here for now.

Within short order, I was proud to find that I had a system which could take requests, do geocoding lookups, save new listings, and display existing listings when a search was made.  But then I read some of the fine print--well, it wasn't *that* fine, but still--and found that Google Maps geocoding engine only accepts 15,000 requests per day from a given IP address, and while I don't know too much about the ins and outs of Google's infrastructure, I ran a test where my app would query my personal server every time I hit a certain URL, and regardless of where the request to my Google AppEngine app originated from--several servers I have access to around the country and the world--the same Google IP address was hitting my servers.

My understanding is that when hitting GAE apps, the user will hit infrastructure close to him or herself, but outgoing HTTP requests all seem to be coming from one proxy, and since I was getting those geocoding requests via HTTP requests, it seemed possible that Google might see them all coming from the same location.  I hope not, but I figured it was worth the challenge to move those requests out to the user layer, in the browser.

So a couple of hours of broken application later, I made it so that all geocoding requests come from the browser, and then the browser queries the SaleRadar system asynchronously, and gets an XML response with either an acknowledgement that the sale was saved, or a list of locations.  Now, I realize that I've got nowhere near 15,000 visitors a day (just me, for now!), but this actually solved a few problems:
  • If (when?) my app becomes uber-popular, I don't have to worry about running out of free Google geocoder requests.
  • If (when?) my app becomes uber-popular, I save a bit on bandwidth and processing, since the browser is doing a pretty good chunk of the work.
  • It made the user interface nicer: no page reloads, since everything is AJAX driven.
  • It had the side-effect of creating an API; if I want, I can publish it so that people can post or search my database from other apps.  I need to add some security if I'm going to go that route, but the option is there.
  • It made it easier to separate out the front-end from the back-end, without having to dive into templates.  Effectively speaking, all user interactions are with a simple HTML/Javascript page, and it's only the AJAX calls that talk to the Python AppEngine code.
Previously, I'd had the Javascript interspersed with my Python, and it got pretty ugly in short order.  I plan to learn Django templating, but suddenly I found I did not need to do so, so I even saved myself another step.

But one thing was troubling me: searching for map coordinates surrounding a given point.  The BigTable datastore system was confounding me with its limitations, and based on chatter in the AppEngine group, I wasn't alone...

No comments: