Find Jobs
Hire Freelancers

Web scraper

$30-250 USD

Completed
Posted about 8 years ago

$30-250 USD

Paid on delivery
I need you to develop some software for me. I would like this software to be developed using Python. Starting at [login to view URL] i need to cycle through every set of cards, record what set the card belongs to (eg Oath of the Gatewatch), download the photo of card, name of card (eg General Tazri), cost (eg 4W), type (Legendary Creature - Human Ally), description (eg When General Tazri enters the battlefield, you may search your library for an Ally creature card, reveal it, put it into your hand, then shuffle your library. WUBRG: Ally creatures you control get +X/+X until end of turn, where X is the number of colors among those creatures), illustrator (eg Chris Rhan), and damage & toughest (eg 3/4). I have used this card for the examples above - [login to view URL] - There may be other fields for different sets, I want to extract ALL data. I want the data output in two formats, Excel / CSV and XML. The web scraper has to be future proof, i want to be able to run this program periodically so as my database is kept up to date with new sets and cards as they are added. A basic user interface would be beneficial but not absolutely necessary as i am happy using the command interface. I will be using this software on a Mac and PC.
Project ID: 10062622

About the project

15 proposals
Remote project
Active 8 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
I already implemented a basic card scraper during the bidding period. You can check out the demo gif here: [login to view URL] .Right now it can get the data from the web and is about 100 lines long. I can complete this project because I have made a similar program to help me play soccermanager.com. It scrapes player data from soccerway.com. The code is available at [login to view URL] . Now my skills are much higher than when I made it. The program will store the data primarily in sqlite tables from which it can be easily be exported to other formats as required. The interface will be totally command line based. All the commands will be arguments to the program. For eg. To update the database, the command will be like: $ mythicspoiler --fetch The program will provide progress data while it does the search which could be turned on and off at demand before the program starts. The biggest hurdle in this project is the extraction of data about the cards from the images only. I have researched a bit and found that most of the cards do not have thesaurus for them. I may have to use some advanced library to do the work of reading the text from the images. Still it is not guaranteed to work as the image quality is not very good. So I cannot guarantee to add ability to read from images. If you know any other sources of getting the information that would be helpful. Feel free to contact me by personal message.
$99 USD in 5 days
5.0 (1 review)
2.0
2.0
15 freelancers are bidding on average $158 USD for this job
User Avatar
Hi there! I have read what you exactly need, however I would like to ask you a few questions. I would call myself a master of what I do, I do work smart and do not rest until I get the job done. Please feel free to ping me anytime so we can have a detailed discussion. If I can deliver I will deliver in best possible way. Thanks
$144 USD in 3 days
4.9 (102 reviews)
8.5
8.5
User Avatar
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi
$205 USD in 5 days
5.0 (141 reviews)
7.6
7.6
User Avatar
Hello Sr., how are you ? Jorge is my name and I have wide experience doing scrapes using different tools. You can visit my Portfolio to know my last jobs regarding these skills, the reviews received from clients and as you can see, a higher percentage of jobs completed. i'm interested to help you on this particular project. What I prefer is to discuss with you more in detail all your terms and I'll give you a better proposal. No more Sr.; contact me if you'd like it. Regards. Jorge. P.S.: I will not accept any award before to discuss with you all the details needed.
$250 USD in 5 days
5.0 (119 reviews)
7.2
7.2
User Avatar
Hi there, I have read the description..I have good web scraping reviews.. I can scrape the website with custom scripts in Python.. Looking forward to discussing details with you,..
$100 USD in 2 days
5.0 (221 reviews)
7.1
7.1
User Avatar
hello , sir : c/c++/python expert worked for samsung & huawei . a sample can be provided before hired hope to get message from you thank you very much .
$200 USD in 10 days
5.0 (23 reviews)
5.3
5.3
User Avatar
Web scraping expert I use python language. My scripts works on windows, mac or linux, but linux is preferably. I can schedule scripts on server if it is required. I have more 100 finished projects (google scraping, facebook scraping, yellow pages, linkedinIn, amazon, webshops and other sites with lists of any items). I can scrape secured and protected sites, my crawlers can enter into login form, emulate ajax requests etc. If site block IP i can use proxy or TOR. I can try avoid captha on site in avtomatic or manual mode. I can export data into json, xml, csv (excel), or any database (mysql, mongodb, mssql, etc). I can develop web-interface for management running script (start, stop, etc), using PHP, HTML, JS.
$222 USD in 4 days
5.0 (12 reviews)
5.0
5.0
User Avatar
Hi there. I can build a scraper with the Scrapy framework. You can run the script in mac or windows, but it would be console-based. CSV and XML are both supported. If you're interested in my offer I will take a closer look at the site and we can work on the details.
$140 USD in 10 days
5.0 (13 reviews)
4.5
4.5
User Avatar
Добрый день. Меня зовут Евгений. Я Python разработчик. У меня есть опыт в парсинге сайтов. Меня заинтересовало Ваше задание. Буду рад помочь.
$55 USD in 7 days
4.9 (8 reviews)
3.4
3.4
User Avatar
Hi I did several similar projects by my own (stocks data cropping etc.) What is the schedule?
$210 USD in 7 days
5.0 (6 reviews)
2.4
2.4

About the client

Flag of RUSSIAN FEDERATION
St Sampson, Russian Federation
5.0
11
Payment method verified
Member since Oct 9, 2014

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.