Jump to content



Photo

Website scraping

images scraping content

  • Please log in to reply
7 replies to this topic

#1 magraw

magraw

    Neowinian

  • Joined: 14-August 05
  • Location: London

Posted 20 July 2013 - 16:41

Hello all,

 

I was hoping for some guidance, any feedback would be greatly appreciated!

 

In short, I have set up a fashion site. On the site people can comment on and like fashion images.

 

For the site to generate content I need to source images from other fashion sites, both the fashion images (e.g. shoes, jumpers etc) and to also include each of their associated content (e.g. price, descriptions and so forth).

 

I was hoping you could tell me the best way to source this content.

 

One method is APIs and I am trying to secure these from major brands. However, this is taking time and also not all sites have an API.

 

I was hoping to 'scrape' these images and their associated content, on a regular basis from selected sites. 

 

My key questions are:

 

1) Is there a tool that can do this? If so, are there tools / companies who are well known / the best at this?

2) What is the likely cost of such a tool?

3) Can you do this manually? i.e. outsource?

4) How much would the outsource model cost?

5) Is there any outsource companies that are best in market? and/or a good price for a starter fashion website?

 

One example scenario might be:

 

- 1000 images per website, with associated content

- 5 websites

- once a month update

 

I look forward to hearing from anyone who can help.

 

Many thanks,

Magraw




#2 Nick H.

Nick H.

    Neowinian Senior

  • Tech Issues Solved: 10
  • Joined: 28-June 04
  • Location: Switzerland

Posted 20 July 2013 - 17:08

I've not been a coder for a while, but the first thing that I think of is this: Do you have permission to take those photos from the sites that you are scraping the images from?

#3 Obry

Obry

    Intaxity.com

  • Joined: 27-September 04
  • Location: Winter Garden, FL
  • OS: Windows 8.1 Enterprise
  • Phone: iPhone 5

Posted 20 July 2013 - 17:17

Unfortunately I don't think it's exactly legal for you to do that. It's copyright violation to do so unless you have express written notice from the owners of the websites you are trying to scrape. If you do get a written permission, then you might be able to do this. However, as this is generally considered gray area (scraping website content) you will have to dig a little deeper and use tools that might not be exactly easy to use (Google is your friend here). I don't think you'll come across some company that actually specializes in such stuff (I could be wrong though)...



#4 Aergan

Aergan

    Neowinian Senior

  • Tech Issues Solved: 6
  • Joined: 24-September 05
  • Location: Staffordshire, UK
  • OS: Windows 8.1 Pro / Server 2012 R2 / Xubuntu 14.04
  • Phone: Sony Xperia Z1

Posted 20 July 2013 - 20:42

You'll not only need permission to send a tonne of non-user traffic (which will skew their reports/stats potentially) but also permission from the copyright holder of each image which may not belong to them.



#5 thatguyandrew1992

thatguyandrew1992

    Neowinian Senior

  • Tech Issues Solved: 3
  • Joined: 22-January 09

Posted 20 July 2013 - 20:55

A google search returned this site

http://mozenda.com/

http://www.visualwebripper.com/

 

A program could be created to scrape the images. The complexity will vary depending on the site being scraped and what you want from each page.



#6 shozilla

shozilla

    Neowinian Senior

  • Tech Issues Solved: 6
  • Joined: 11-January 09

Posted 20 July 2013 - 21:01

Similar to my old website that I used to buy the licensed packages of the photos then display them on my site for the visitors to see. You can buy licensed packages of the items you want from the fashion based market or whoever you buy from so you will be able to display the items on your site and update whenever you can as long as they have new items coming up (days or months) ... Make sure to read the agreements and/or terms before purchasing or getting any of the images otherwise you will be screwed and/or ended up getting sued for copyright violations. Best thing to do is contact the sellers or market owners about the items you want to use for permissions... and make sure both ends have no problems with the said agreements. If you steal the images from the sites, someone will visit and check for any violations you make, if so, they will contact you or sue you for that. You can use API with linked remote-based images/contents that can be updated automatically by the markets which you may not have to do a thing.

#7 Moska

Moska

    Resident One Post Wonder

  • Joined: 21-July 13

Posted 21 July 2013 - 13:55

@Neowinian

 

Hi, You can try this http://scrape4me.com/

It's free and very simple to use. Let me know if you need help with it



#8 Original Poster

Original Poster

    C++ n00b

  • Tech Issues Solved: 1
  • Joined: 15-July 08
  • Location: my room
  • OS: windows 7, backtrack 5, OSx 10.6

Posted 25 July 2013 - 04:20

ok what you are requesting is illegal is you are not given an API and are scraping you are most likley violating many laws 





Click here to login or here to register to remove this ad, it's free!