• 0

some errors in a simple Python parser-script - that makes use of BS4


Question

 

 

hello dear all

 

just to do some approach to python i have tried to get some data from yahoo .- with the following script ..

 

note the while look should fit the aim that the data were fetched constantly - guess that this part is working - but i get some errors.

 

 

import bs4
import requests
from bs4 import BeautifulSoup



def parsePrice():
r=requests.get('http://finance.yahoo.com/quote/FB?p=FBoup=bs4.BeautifulSoup(r.text,"xml")
price=soup.find_all('div',{'class':'My(6px) Post(r) smartphone_Mt(6px)'})[0].find('span').text
return price



while True:
print('the current price_ '+str (parsePrice()))




 

 

 

but there is missing some thing . getting errors all the way - in my ATOM - Editor 

 

 

Traceback (most recent call last):
File "C:\Users\Kasper\Documents\_f_s_j\_mk_\_dev_\bs\yahoo_finance.py", line 14, in <module>
print('the current price_ '+str (parsePrice()))
File "C:\Users\Kasper\Documents\_f_s_j\_mk_\_dev_\bs\yahoo_finance.py", line 10, in parsePrice
price=soup.find_all('div',{'class':'My(6px) Post(r) smartphone_Mt(6px)'})[0].find('span').text
IndexError: list index out of range

 

 

any idea what goes wrong here 

1 answer to this question

Recommended Posts

  • 0

The problem is that the requested URL uses a bunch of javascript to render the page in html when the page is requested, so you need to set the bs4 parser to 'html.parser' instead of xml. But! This is a webpage rendered for display, not parsing. The div classes are very convoluted and may differ based on your request headers (which you never set). Because of the html, I find it easier to just parse it using string methods rather than bs4. After that, you can just convert it to a json dictionary and return whatever you want. Here's a quick example that prints the price quote every minute with a timestamp:

 

import requests
import json
from datetime import datetime
import time


def priceParse(stock):
    url = 'https://finance.yahoo.com/quote/' + stock
    html = requests.get(url).text
    json_str = html.split('root.App.main =')[1].split('(this)')[0].split(';\n}')[0].strip()
    data = json.loads(json_str)['context']['dispatcher']['stores']['QuoteSummaryStore']
    latest_price = data['price']['regularMarketPrice']['fmt']
    market_state = data['price']['marketState']
    timestamp = datetime.now().strftime("%m/%d/%Y, %H:%M:%S")
    printed_str = stock + ' — $' + latest_price + ' (market state: ' + market_state + ') — ' + timestamp
    print(printed_str)
    return


while True:
    priceParse('FB')
    time.sleep(60)

 

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • I disabled the optical camera in device manager.. leaving only the IR. This has fixed the issue for me...but only because I never use the optical camera. After each monthly update re-enable the optical just to see if it's fixed...but nope! It's annoying though how this issue hasn't been acknowledged by Microsoft.
    • Linux is a different kettle of fish to macOS and Windows, if it ran the software I required, I may have looked at it, instead of the Mac, also the Mac is a pretty powerful machine that uses less energy than x86 machines. I never in my widest dreams thought I would ever buy a Mac, the price and restrictions of the hardware, I always liked machines that I could update internally, one reason why I never liked laptops. But here I am, a nice little Mac mini M2 pro. I doubt i will replace it for a long time, if I ever do, it does what I need.
    • 106 years ago! A comic strip from 1919 predicted — eerily and accurately — what would happen if our phones fit into our pockets.  W. K. Haselden’s ‘The Pocket Telephone: When Will it Ring?’ was published in “The Mirror” when barely 1/3rd of American homes even had telephones. (A double irony: most of us are viewing this on our “pocket phones”.)
    • It's a feature, not a bug: Windows Hello face unlock no longer works in the dark by Usama Jawad Windows Hello is a security mechanism offered by Microsoft in compatible Windows PCs that allows customers to use biometric authentication mechanisms such as facial or iris recognition, fingerprint, or a PIN to access their accounts. Although it's a great feature when it works correctly, that is not always the case. Now, Microsoft has changed the way this feature works, and people aren't exactly pleased. As spotted by Windows Central, Windows Hello hasn't been working as people would expect since at least April's Patch Tuesday update. Since then, the mechanism requires both an infrared (IR) sensor and a color webcam to function correctly. Previously, Windows Hello relied on IR sensors to create a 3D facial map, just like the iPhone. IR sensors work quite well in low-light environments, but Microsoft does not allow customers to rely on them anymore due to a patch for a security vulnerability. It is important to note that most webcams don't illuminate the environment on their own and rely on external light. This essentially means that Windows Hello simply doesn't work in low-light environments for many users. In our own testing, Windows Hello seems to be working in low warm light, shown below: However, it now fails to work in an even dimmer environment, such as that with red light: That said, it seems that Microsoft has not implemented the change perfectly. Users have reported that if they disable their webcam through Device Manager, Windows Hello falls back to using IR sensors only and continues to function the way it used to. While that may act as a workaround for users right now, it'll be interesting to see if Microsoft patches this loophole down the line, too; it certainly wouldn't be surprising now that it's more widely known. Source: Windows Central
  • Recent Achievements

    • Week One Done
      patrickft456 earned a badge
      Week One Done
    • One Month Later
      patrickft456 earned a badge
      One Month Later
    • One Month Later
      Jdoe25 earned a badge
      One Month Later
    • Explorer
      Legend20 went up a rank
      Explorer
    • One Month Later
      jezzzy earned a badge
      One Month Later
  • Popular Contributors

    1. 1
      +primortal
      642
    2. 2
      ATLien_0
      278
    3. 3
      +FloatingFatMan
      172
    4. 4
      Michael Scrip
      156
    5. 5
      Steven P.
      132
  • Tell a friend

    Love Neowin? Tell a friend!