QUOTE(KGSodie @ Aug 17 2007, 09:42 AM)

I don't know if anyone would be interested in the information, but I've been working on some tools to 'mine' the data on the USCIS site. It's all public information, after all, and I've been curious how many petitions get filed each day, and how many are I129F petitions.
I have the code all done but the USCIS has thrown me a curve - they have monitors in place that check how many requests are coming from a given IP (or possibly gateway), and they 'shut off' access for 2 or 3 days if you exceed whatever the limit is. There is a flood check for a short duration, and after I slowed my requests down (one lookup every 15 seconds) I hit a longer duration monitor that also shut me down.
So my next try will be to slow my code WAY down (maybe one lookup every 2 to 5 minutes), but I have already determined that the average number of petitions received by the California Service Center is around 4000 (or more) (my code has a binary search component that can determine the number of petitions in about 14 searches). To determine which are I129F petitions (or any other type) I have to query each individual petition. Once I know which ones are I129F, I log the numbers so that I can watch for status changes on subsequent days.
So if I query 1 case every 2 minutes, it will take 8000 minutes, or 134 hours, or nearly 6 full days. And that's just to locate the receipt numbers for a single day. To scan all of July (our filing month) will take about 117 days (21 fiscal days in July, or so)! I would eventually like to give this code to VJ so they don't have to rely on users to input timelines, and they would gain access to a ton of more petitions. I have already contacted VJ and they are interested - if I can get it to work.
Why do all of this? Because I can, and I want to. It's an interesting technical challenge. Would there be anyone out there reading this who would like to donate some of their computer time? I could provide you the code (it's written in python) along with the instructions, and we could split up the days in July of 2007, in order to locate the petitions we're interested in.
I've also requested of the USCIS that this data be made available in an off-line form so I don't have to query it online so slowly. If they agree, and I'm not holding my breath, then this would all be moot, but I thought I'd see if there is any interest, other than my own!
Good luck to you, but you realise they are a paper system at present not computer , for example they still have my N-400 as pending from May of 2006, I got it months ago, but if you get anywhere it would be interesting