• 0

Need with with Imgur program


Question

Hey guys,

I wanted to work on a new project just for fun, so I thought a program that downloaded imgur galleries would be cool. I am working on it, but the program will run through duplicates after 51 images usually.

I've looked at all i can and can't find the problem. My guess is that it's the server resetting me. I wanted a challenge so i tried to get past it. I tried creating a new url connection each time, I also created a seperate class that did the downloading and creating a new object of that class everytime but it didn't work.

Thoughts?

Source

http://shayconcepts.com/programming/ImgurGalleryDownloader/imgurDL.zip

Link to comment
https://www.neowin.net/forum/topic/1110175-need-with-with-imgur-program/
Share on other sites

19 answers to this question

Recommended Posts

  • 0

You are parsing the HTML page. Don't do that. Use imgur's API instead - from the documentation, you can just append ".xml" to any gallery page and it returns the gallery as an XML file.

In general, don't parse HTML pages unless there is no other way. It's not easy and it can change any day, making your program useless.

  • 0
  On 04/10/2012 at 20:30, Aethec said:

You are parsing the HTML page. Don't do that. Use imgur's API instead - from the documentation, you can just append ".xml" to any gallery page and it returns the gallery as an XML file.

In general, don't parse HTML pages unless there is no other way. It's not easy and it can change any day, making your program useless.

Thanks! I'll take a look.

EDIT: it seems it only returns 56 images though :/

  On 05/10/2012 at 01:51, Dr_Asik said:

I learned about regex and it has worked really well for me! I'll take a look at your link and find a better way to parse HTML. Thanks!

  • 0
  On 05/10/2012 at 04:14, thatguyandrew1992 said:
I learned about regex and it has worked really well for me! I'll take a look at your link and find a better way to parse HTML. Thanks!
If you really need to parse HTML, you should use a proper HTML library; for Java there is HtmlUnit and Jsoup at least. Better would be to not parse any HTML and use imgur's API as Aethec suggested.
  • 0
  On 05/10/2012 at 06:05, Dr_Asik said:

If you really need to parse HTML, you should use a proper HTML library; for Java there is HtmlUnit and Jsoup at least. Better would be to not parse any HTML and use imgur's API as Aethec suggested.

But when using the API, I append the .xml but it doesn't show ALL the images in the gallery

  • 0
  On 12/10/2012 at 04:27, thatguyandrew1992 said:

Ok guys, I need some more help. How do I get the xml of an album? They are apparently different than galleries! The xml page isn't coming up. Or if there isn't an xml, what is recommended to get the images?

Ex

http://imgur.com/a/HKtAR/

It's all there in the API documentation: http://api.imgur.com/resources_anon#album

Example: http://api.imgur.com/2/album/HKtAR

  • 0
  On 12/10/2012 at 14:05, ZakO said:

It's all there in the API documentation: http://api.imgur.com...rces_anon#album

Example: http://api.imgur.com/2/album/HKtAR

Thanks! I'm learning how to use APIs, I know this is all obvious. I thought I had looked through this. ill be sure to look extra hard!, >.<

This topic is now closed to further replies.
  • Posts

    • Is it though?  I built a new rig a few months ago and it was literally impossible to get one without RGB, but within 10 minutes of setting it up, I turned all that crap off.  It was REALLY distracting, and who needs additional heat INSIDE a PC? It's popular on YouTube for sure, it's neat looking and whatnot, but it's about as practical as a coffee cup with a hole in it. As for the price, a non-enthusiast would just see something priced way above what they can get from a retailer brand new...
    • RollBack Rx Pro 12.9 Build 2710971022 by Razvan Serea RollBack Rx is a robust system restore utility that enables home users and IT professionals to easily restore a PC to a time before certain events occurred. In essence, it turns your PC into a Instant Time Machine. Regardless of what happens to your PC your can quickly and easily restore your PC to a previous time. Making it easy to rescue you from any PC disaster - saving time, money and PC trouble. Windows System Restore only restores Windows system files and some program files. In addition, if Windows crashes to a point were Windows itself can not boot up (ie. BSOD*) you would not be able to access your Windows System Restore points. In contrast, the RollBack Rx technology works at the sector level of the hard drive and restores everything! - right down to the last byte of data. It sits below Windows. So even if Windows crashes, there’s a sub-console (mini OS) that boots prior to windows. This allows you to access Rollback Rx and go back to a point in time when your system was working trouble-free. Key Features Go back to any previous point in time within seconds. Go back minutes, hours, days, weeks, or even months to any previous snapshot. Does not affect computer performance, uses minimal system resources. Supports unlimited snapshots. Creates a complete system snapshot without having to restart the system. Reverse any system crash within seconds (even if Windows cannot startup). Back out of any failed program, OS updates, and botched updates. Recover from any malware or virus attack within seconds. Works with VMWare and Virtual Machines, both as a host or within the virtual machine as a client. Supports Multi-boot, Multi OS workstations. Lock snapshots to prevent deletion. Intuitive GUI based snapshot manager. Explore, browse, and retrieve files and folders from any snapshot. Drag and drop them into your active system. Roll backwards as well as forwards to any available system snapshot. Allows users to safely test any software. Fast, 100% complete uninstaller. Retrieve files from a crashed PC, even if Windows cannot boot. Access control – manage levels of multiple user and administrative privileges. Automatically schedule snapshots to be taken on a fixed schedule or upon execution of specific files (ie. setup.exe) as well as manually. 256 bit AES snapshot encryption. Prevent unauthorized data theft in case of a stolen laptop. Group Management and Enterprise Network Administration Control (FREE utility). Comes with Stealth Mode where you can hide the RollBack Rx tray icon and splash screen (seen during bootup) Change the startup hotkey for sub-console access (default is HOME). Built-in snapshot defragmenter which will optimize system resources and recover free space. Option to keep files and folders unchanged when you roll-back. Advanced setup configuration wizard for system administrators which will set deployment options and predefined RollBack Rx settings. Offers detailed program operation logging. Supports all industry-standard deployment options including silent installations and pre-installation configuration. Explore RollBack Rx Pro with a 14-day trial, fully functional on Windows 11, 10, 8, and Windows 7 SP1** (32 and 64-bit). RollBack Rx Pro 12.9 Build 2710971022 changelog: General Add PnpLockdown in shieldm.inf Fix registry exclusion problem in Windows 11 24H2 release Add detailed logging for file filter driver Add detailed logging for Windows update Add time stamp to kernel drivers Change kernel driver and Win32 IRP structure Other small bug fixes / typos reported through tech support Endpoint Manager Add client report dashboard Add sound effect when receiving a EPM message. Keep EPM message history Fix bug that oversized Windows symbol files cannot be downloaded Download: RollBack Rx Pro 12.9 | 61.0 MB (Shareware) View: RollBack Rx Home Page Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • Universal Media Server 14.12.1 by Razvan Serea Universal Media Server is a DLNA-compliant UPnP Media Server. UMS was started by SubJunk, an official developer of PMS, in order to ensure greater stability and file-compatibility. The program streams or transcodes many different media formats with little or no configuration. It is powered by MEncoder, FFmpeg, tsMuxeR, AviSynth, MediaInfo and more, which combine to offer support for a wide range of media formats. Because it is written in Java, Universal Media Server supports all major operating systems, with versions for Windows, Linux and Mac OS X. To see a comparison of popular media servers, click here. Universal Media Server 14.12.1 changelog: General Added status page to readme Fixed videos not being marked as fully played (#5373) (thanks, @Fredo1650!) Fixed adding YouTube channels from handle URLs (URLs with @ in them) Fixed handling special characters on Linux (#5100) (thanks, @LaTeteDansLesEtoiles!) Fixed directory browsing crash (#5189) (thanks, @jt-gilkeson!) Fixed FFmpeg on Linux x86_64 and arm64 (#5465) (thanks, @KanjiMonster!) Fixed logspam like "Could not hydrate device or its services from descriptor" (#5292) (thanks, MTOakey!) Fixed broken YouTube video playback Fixed web interface E2E testing on CI using outdated code because of overeager caching Fixed broken video playback when burning subtitles to H.265 via FFmpeg (#5486) Improved logging Translation updates via Crowdin Chinese (Simplified) (59%) (thanks, 無情天!) Dutch (41%) (thanks, Matthias!) Hungarian (86%) (thanks, Zoltán Rózsa!) Japanese (69%) (thanks, Yukihuru!) Download: Universal Media Server 14.12.1 | 203.0 MB (Open Source) Download: Other operating systems View: Universal Media Server Website | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • You sign your rights to reddit when you write on their platform. Free labour for them to make money. The AI companies should also take advantage of that free labour.
    • Brave 1.79.123 by Razvan Serea Brave Browser is a lightning-fast, secure web browser that stands out from the competition with its focus on privacy, security, and speed. With features like HTTPS Everywhere and built-in tracker blocking, Brave keeps your online activities safe from prying eyes. Brave is one of the safest browsers on the market today. It blocks third-party data storage. It protects from browser fingerprinting. And it does all this by default. Speed - Brave is built on Chromium, the same technology that powers Google Chrome, and is optimized for speed, providing a fast and responsive browsing experience. Brave Browser also features Brave Rewards, a system that rewards users with Basic Attention Tokens (BAT) for viewing opt-in ads. This innovative system provides an alternative revenue model for content creators and a way to support the Brave community. Brave 1.79.123 changelog: Leo Improved citation UI. (#45761) General [Security] Fixed missing DDNS navigation throttle for subframes as reported on HackerOne by newfunction. (#46703) Fixed crash which occurred when clicking on the “View site information” icon in the address bar while having “Don’t allow sites to scroll and zoom shared tabs” enabled. (#46566) Fixed crash which occurred with the “Save autofill” prompt in certain cases. (#45546) Upgraded Chromium to 137.0.7151.104. (#46712) (Changelog for 137.0.7151.104) Download: Brave Browser 64-bit | 1.2 MB (Freeware) Download: Brave Browser 32-bit View: Brave Homepage | Offline Installers | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
  • Recent Achievements

    • Week One Done
      somar86 earned a badge
      Week One Done
    • One Month Later
      somar86 earned a badge
      One Month Later
    • Apprentice
      Adrian Williams went up a rank
      Apprentice
    • Reacting Well
      BashOrgRu earned a badge
      Reacting Well
    • Collaborator
      CHUNWEI earned a badge
      Collaborator
  • Popular Contributors

    1. 1
      +primortal
      510
    2. 2
      ATLien_0
      260
    3. 3
      +Edouard
      190
    4. 4
      +FloatingFatMan
      175
    5. 5
      snowy owl
      133
  • Tell a friend

    Love Neowin? Tell a friend!