xml parsing - Looking for an Open Source Web Crawler that can crawl API requests and parse XML into csv -

-

i'm looking webcrawlers crawl through api , parse xml xml or csv file.

i've been playing around requests api feeds great if didnt have manually , use automatically , edit data later.

for example using api site called eventful, can request "?xml feed?" of data

http://api.eventful.com/rest/events/search?app_key=lksbnc8mgtjd4wc5&location=pittsburgh&date=future

if inspect link can see there ton of xml data sent back.

i thought since xml data broken down elements wouldn't difficult ask crawler handle sorting (e.g city element send data city field in csv document)

i'm wondering if has used existing opensource web crawler crawl apis , relate parsed data excel format....

i looked nutch couldnt find reference in documentation sorting xml return excel document based on elements returned api feed.

has done before , can refer program. specifics helpful.

we @ http://import.io/ have free solution similar mozenda, build api using our web browser , can upload api our servers , use free. offer crawler , various other features. check out , see think :)

p.s work import.io if didn't already.


Comments

Post a Comment

Popular posts from this blog

java - SNMP4J General Variable Binding Error -

-
i trying use snmp4j snmp bulkget. when ever use snmp4j make call returned responseevent error says "general variable binding error" , data returned equal null. to debug: print out, console, exact oid , version number using in snmp4j. use printed out data snmpwalk on command line. valid results. know sending snmp4j correct pdu correct oid, version number, max repititions, etc. i have used snmp4j , code wrote succesffuly monitor other devices years. don't know different time. leaving me stumped. why "general variable binding error?" causes error? ideas debug? can reproduced? if so, first action should using wireshark or microsoft network monitor capture network packets. my guess agent gives generr response, not surprising, http://www.ietf.org/rfc/rfc1157.txt
Read more

sql server - python to mssql encoding problem -

-
Image
greetings by using pymssql library, want write data mssql database encounter encoding issues. here sample code write db: # -*- coding: utf-8 -*- import _mssql .... connection info data here .... def mssql_connect(): return _mssql.connect(server=host, user=username, password=pass, database=db, charset="utf-8") con = mssql_connect() insert_ex_sql = "insert mydatabsae (id, programname, programdetail) values (1, 'test characters ÜŞiçÇÖö', 'löşüiiğĞü');" con.execute_non_query(insert_ex_sql) con.close() sadly data written db corrupted: the collacation of mssql db is: turkish_ci_as how can solved? here possible solution : the key insert_ex_sq.encode('your language encoder') . try instead: con.execute_non_query(insert_ex_sq.encode('your language encoder'))
Read more

windows - Python Service Installation - "Could not find PythonClass entry" -

-
i working on building python service. inducing error, not sure inducing it. ps c:\...> python .\file.py debug debugging service myservice - press ctrl+c stop. error 0xc00000f4 - not find service's pythonclass entry in registry error 1814 - specified resource name cannot found in image file. error 0xc0000080 - not locate module name in python class string (ie, no '.') this error coming pythonservicemessages.mc in pythonwin32. it turns out error result caused exception being thrown , python script dying off.
Read more