• 0

vscode: first script-test after a quick installation - some minor issues


Question

hello dear fellows here at neowin, 

 

vscode: first script-test after a qucik installation - some minor issues

on a MX-Linux version 19.1 i have installed VSCodium 1.43.2

Version: 1.43.2

Commit: 0ba0ca52957102ca3527cf479571617f0de6ed50

Date: 2020-03-24T21:03:16.125Z

Electron: 7.1.11

Chrome: 78.0.3904.130

Node.js: 12.8.1

V8: 7.8.279.23-electron.0

OS: Linux x64 4.19.0-6-amd64

 

i have Python installed - unfortunatly not with venv - but globally .

to test the whole system i just run a little testscript.

btw - to setup with venv i will take care later the weekend. Now at the moment i only will test the system.

 

import requests
from bs4 import BeautifulSoup
import pandas as pd
page1 = requests.get('https://en.wikipedia.org/wiki/Peths_in_Pune').text BeautifulSoup(page1, 'lxml')
table = soup1.find('table',{'class':'wikitable sortable'})
#table
table1=""
for tr in table.find_all('tr'):
    row1=""
    for tds in tr.find_all('td'):
        row1=row1+","+tds.text
    table1=table1+row1[1:]
row1

 

see the output

 

 

 

^

SyntaxError: unexpected EOF while parsing

martin@mx:~

$ /usr/bin/python3 /home/martin/dev/python/test.py

File "/home/martin/dev/python/test.py", line 2

^

SyntaxError: unexpected EOF while parsing

martin@mx:~

$ /usr/bin/python3 /home/martin/dev/python/test.py

Traceback (most recent call last):

File "/home/martin/dev/python/test.py", line 5, in <module>

soup1 = BeautifulSoup(page1, 'lxml')

File "/usr/lib/python3/dist-packages/bs4/__init__.py", line 196, in __init__

% ",".join(features))

bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?

martin@mx:~

$ /usr/bin/python3 /home/martin/dev/python/test.py

Traceback (most recent call last):

File "/home/martin/dev/python/test.py", line 5, in <module>

soup1 = BeautifulSoup(page1, 'lxml')

File "/usr/lib/python3/dist-packages/bs4/__init__.py", line 196, in __init__

% ",".join(features))

bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?

martin@mx:~

$ python3 -V

Python 3.7.3

martin@mx:~

$ python3 -V

Python 3.7.3

martin@mx:~

 

 

...any idea - and thoguts regarding this error

 

Edited by tarifa

3 answers to this question

Recommended Posts

  • 0

your linux missing also relevant libraries for both bs4 lxml

pip can also install missing

missning things inside python or python3


pip install lxml
pip install beautifulsoup4


pip3 install lxml
pip3 install beautifulsoup4


an other hint about all of this is too use

from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc, 'html.parser')

print(soup.prettify())

 

vscode use some old packages?

 

i did take an look at vscode for Windows but just for see what is inbuilded inside vscode i did see 300 several different projects / programs from github

 

this also causes bugs to occur if you have so many different sources that you need to update for the entire vscode

 

you probably could have solved the problems by building vscode from the source code but other problems are surely occurring in the meantime

Edited by Christopher Andreason
add some extra info
  • 0
  On 04/04/2020 at 01:11, Christopher Andreason said:

 

Expand  

hello dear Christopher, 

 

first of all - many many thanks for the quick reply - this thread is the result of a headstart into Python with VSCode - since ATOM does not seem to be able to do all what i want to do.

So i try to setup VScode to work with Python. 

 

  Quote
your linux missing also relevant libraries for both bs4 lxml

pip can also install missing

missning things inside python or python3


pip install lxml
pip install beautifulsoup4


pip3 install lxml
pip3 install beautifulsoup4


an other hint about all of this is too use

from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc, 'html.parser')

print(soup.prettify())

 

vscode use some old packages?

i did take an look at vscode for Windows but just for see what is inbuilded inside vscode i did see 300 several different projects / programs from github

this also causes bugs to occur if you have so many different sources that you need to update for the entire vscode

you probably could have solved the problems by building vscode from the source code but other problems are surely occurring in the meantime

Expand  

 

Yes i guess  that you are right - 100%  - i am missing some packages  - and sure thing - VScode does not work with venv - that said - think i have to do a correct and good setup.

 

Well - Christopher i have read some and have seen a whole bunch of tutorials on the setup virtual environment for Python

 

- in VS Code - best practices for using Virtualenv 
- and besides for ATOM ...  - see above. 

i am very very glad that you have posted and give me some important hints: 

 

 

  Quote


it semms like the package which i installed and the project's environment i am currently using is different.
the package which i have  installed was in the global environment: " c:\program files\python37\lib\site-packages ", and well it seems like i looks like using the environment different(maybe the environment related to the project): This said i try ton identify this by look the right left coner shows which environment i am currently using now.

Expand  

 

So i guess that i have to using virtualenv to isolate my projects and then i store pip dependencies in requirements.txt
As we develop, we install, remove and upgrade packages, the list of dependencies in your project differs from requirements.txt

but how can i do it right - how to setup venv the right way - 
a. on win 10 and 
b. on linux ? 

i have setup a python-development-environment on a Windows 10 machine and on a MX-Linux-machine. 
i guess that i set up the machine /ATOM badly - any and all help greatly appreciated. 

 

 

To do some steps into the direction of a decent and correct installation of VSCode and the correct interaction with Python i have written down some ideas ..: 


note: the setup of Python with global mode is weird so weird. - At leaset it seems to be so - if we have a look at this thread.  

 regarding the set up and usage of virtual environment in VSCode 

I recently have read an article on using Virtual Environments for Python projects.
https://towardsdatascience.com/python-virtual-environments-made-easy-fe0c603fe601
 
and this one Comparing Python Virtual Environment tools
https://towardsdatascience.com/comparing-python-virtual-environment-tools-9a6543643a44
 
guess, that have to take care how i setup python on my linux-machine. 

Comparing installed pip packages with requirements.txt :: So if we are using virtualenv to isolate our projects and then subsequently we store pip dependencies in requirements.txt.  As we develop, we install, remove and upgrade packages, the list of dependencies in your project differs from the so called requirements.txt

Currently Installed Packages: To list what are the packages that are actually installed,  we can run

$ pip freeze

 

Compare Differences: A simple comparison of requirements.txt and pip freeze will fail because the packages are in different order. 
sort both of the output then compare them with pip

to sume up: some of the best Practices are the following: 

- Always make sure requirements.txt reflects the actual dependencies
- Pin package dependencies - use the exact version
- Preferably track only the top level dependencies in your requirements.txt
- Update dependencies periodically  - year but how
- Consider using pip-compile and pip-sync to manage your dependencies. Use pur to automatically update your top level dependencies in requirements.txt

 
I am starting to work on VS-Code using venv: In my project folder I guess that i have to create venv folder.

python -m venv venv /path/to/new/virtual/environment

but when i run in VS Code the command select python interpreter my venv folder is not shown. 

to make sure that i do all okay i try the following steps
to make my virtual interpreter in VS Code visible? i 

 

1. just go to File > preferences > Settings  - afterwards i

2. click on Workspace settings.

3. Under Files: Association, we will find Edit in settings.json , Well i click on that.
4. Update "python.pythonPath": "my_venv_path/bin/python" under workspace settings. 

(For Windows): Update "python.pythonPath": "my_venv_path/Scripts/python.exe" under workspace settings.  And subsequently

5. Restart VSCode incase if it still doesn't show the venv.


another option to show virtual environments in vs code: 

go to the parent folder in which venv is there through command prompt.
Type code . and Enter. [Working on both windows and linux for me.]

That should also show the virtual environments present in that folder.

In one workspace folder named Python need to adde all my other projects. 

to spell it out clearly: 

- I would have to have only one venv for the whole workspace folder Python. 
- i add each subfolder in Python folder as a workspace project like Project1, Project2, Project3, Project4, Project5, Project6  etc. 

In that Project folder I created venv environment and edited settings.json for workspace with this "python.venvPath": "venv" .

 

Now, for every new project I will create new workspace and inside that folder goes venv folder which will be automatically recognized.

 

+------------------------+
|                        |
|                        |
|     python-workspace   |
|     ....-folder        |
|                        |
+----------+-------------+
           |
           |
           |              +----------------------+
           |              |                      |
           +--------------+     Project1         |
           |              |                      |
           |              +----------------------+
           |
           |              +----------------------+
           |              |                      |
           +--------------+     Project2         |
           |              |                      |
           |              +----------------------+
           |
           |              +----------------------+
           |              |                      |
           +--------------+     Project3         |
           |              |                      |
           |              +----------------------+
           |
           |              +----------------------+
           |              |                      |
           +--------------+     Project4         |
           |              |                      |
           |              +----------------------+
           |
           |              +----------------------+
           |              |                      |
           +--------------+    Project5          |
           |              |                      |
           |              +----------------------+
           |
           |              +----------------------+
           |              |                      |
           +--------------+   Project6           |
                          |                      |
                          +----------------------+

 

Christropher, how do you like this idea!?

 

additional: some question regarding the practical use of VScode for running and testing scripts. 


I want to be able to export the data I have scraped as a CSV file. My question is how do I write the piece of code which outputs the data to a CSV?

 

Christopher: the question is: can we run in VSCode the code below - and have a closer look at the output? does VSCode execute the code - and store the file somewhere on the machine!?

 

 

import csv ; import requests
from bs4 import BeautifulSoup 

outfile = open('career.csv','wne='')
writer = csv.writer(outfile)
writer.writerow(["job_link", "job_desc"])

res = requests.get("http://implementconsultinggroup.com/career/#/6257").textautifulSoup(res,"lxml")
links = soup.find_all("a")

for link in links:
     if "career" in link.get("hrefd 'COPENHAGEN' in link.text:
        item_link = link.get("href").strip item_text = link.text.replace("Viewtion","").strip()
        writer.writerow([item_link, item_text])
        print(item_link, item_text)
outfile.close()

 

 

We now should be able to run this in VScode ( and yes: i do not think that we need a fully fledged IDE as PyCharm)  - i guess that we  can now open the py file and run it nicely with the shortcut Ctrl+Shift+B (Windows) or Cmd+Shift+B (Apple)


i have done a search on the net - there are quite some extensions for running python:

 

 

Official python extension: This is a must install.

Increadibly useful for all sorts of languages, not just python. Would highly reccomend installing.


AREPL: Real-time python scratchpad that displays your variables in a side window.  I'm the creator of this so obviously I think it's great but I can't give a unbiased opinion

Wolf: Real-time python scratchpad that displays results inline

 

 

And -if we use the integrated terminal we can run python in there and not have to install any extensions.

 

  [1]: https://marketplace.visualstudio.com/items?itemName=ms-python.python
  [2]: https://marketplace.visualstudio.com/items?itemName=formulahendry.code-runner
  [3]: https://marketplace.visualstudio.com/items?itemName=almenon.arepl
  [4]: https://marketplace.visualstudio.com/items?itemName=traBpUkciP.wolf
  [5]: https://marketplace.visualstudio.com/items?itemName=donjayamanne.jupyter
 

 

dear Christopher - many many thanks for your reply  - and for the idea sharing. 

 

i am very glad to be here - and to be able to share ideas in this thread.

 

have a great day

 

regards 

tarifa

Edited by tarifa
  • 0

 

hello dear Christopher 

 

i reworked the things in the VSCode - runned the following code ... :  (note: with all the necessary plugins and extensions loaded in python) and got back the following 

in the terminal - see below... 


import requests
from bs4 import BeautifulSoup
import re
import csv
from tqdm import tqdm


first = "https://europa.eu/youth/volunteering/organisations_en?page={}"
second = "https://europa.eu/youth/volunteering/organisation/{}_en"


def catch(url):
    with requests.Session() as req:
        pages = []
        print("Loading All IDS\n")
        for item in tqdm(range(0, 347)):
            r = req.get(url.format(item))
            soup = BeautifulSoup(r.content, 'html.parser')
            numbers = [item.get("href").splitsplit("_")[0] for item in soup.findAll(
                "a", href=re.compile("^/youth/volunteering/organisation/"), class_="btn btn-default")]
            pages.append(numbers)
        return numbers


def parse(url):
    links = catch(first)
    with requests.Session() as req:
        with open("Data.csv", 'w', newline="", encoding="UTF-8") as f:
            writer = csv.writer(f)
            writer.writerow(["Name", "Address", "Site", "Phone",
                             "Description", "Scope", "Rec", "Send", "PIC", "OID", "Topic"])
            print("\nParsing Now... \n")
            for link in tqdm(links):
                r = req.get(url.format(link))
                soup = BeautifulSoup(r.content, 'html.parser')
                task = soup.find("sectionass_="col-sm-12").contents
                name = task[1].text
                add = task[3].find(
                    "i", class_="fa fa-location-arrow fa-lg").parent.text.strip()
                try:
                    site = task[3].find("a", class_="link-default").get("href")
                except:
                    site = "N/A"
                try:
                    phone = task[3].find(
                        "i", class_="fa fa-phone").next_element.strip()
                except:
                    phone = "N/A"
                desc = task[3].find(
                    "h3", class_="eyp-project-heading underline").find_next("p").text
                scope = task[3].findAll("span", class_="pull-right")[1].text
                rec = task[3].select("tbody td")[1].text
                send = task[3].select("tbody td")[-1].text
                pic = task[3].select(
                    "span.vertical-space")[0].text.split(" ")[1]
                oid = task[3].select(
                    "span.vertical-space")[-1].text.split(" ")[1]
                topic = [item.next_element.strip() for item in task[3].select(
                    "i.fa.fa-check.fa-lg")]
                writer.writerow([name, add, site, phone, desc,
                                 scope, rec, send, pic, oid, "".join(topic)])


parse(second)

 

 

see the output in the terminal - the question is: where are the results are stored!?

 

image.thumb.png.605a091d42e4cc0e2996ed6ec8f7017c.png

 

 

Should i do some more settings in VSCode !?  Do i need more plugins?

 

look forward to hear from you 

 

regards 

tarifa

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • How to take HDR screenshots on your iPhone running iOS 26, and other new features by Aditya Tiwari Apple has leveled up the screenshots game on its latest iPhone software update, iOS 26. Alongside the updated Photos app, Compact mode in Safari, and new wallpapers, iOS 26 comes with a redesigned screenshot editor, featuring the controversial Liquid Glass system design. The iOS 26 update brings new features and changes that let you customize your screenshot experience in a better way. For starters, it displays the captured screenshot with rounded corners, shifting away from the rectangle on iOS 18. Turn on HDR screenshots on iOS 26 A highlight of the updated screenshots features on iOS 26 is that you can take screenshots in HDR (High Dynamic Range). Apple has added a new Screen Capture page in the Settings app that lets you do that. You can follow these steps to enable HDR screenshots on iOS 26: Open the Settings app on iOS 26. Go to General > Screen Capture. Now, select the HDR option under the Format section. Press the Home + Volume Up button combination to take a screenshot. Next time you take a screenshot on your iPhone, it will be captured with full dynamic range and stored in HEIF format. In other words, if you take a screenshot of a photo and video available in HDR, the feature will preserve those details. On the other hand, the SDR quality saves screenshots in PNG format and offers more compatibility. As shown in the image below, you can check whether a screenshot is captured in HDR or not by swiping up on a screenshot in the Photos app to open its details. Record your iPhone screen in HDR on iOS 26 Not just screenshots, the HDR option also works for screen recordings. Such files are saved in HEVC HDR10 format, whereas screen recordings taken in SDR mode are saved in HEVC. After switching to the HDR options using the steps above, you can use the same old steps to screen record your iPhone; the option is available in the Control Center. That said, you can do a lot more to improve your screenshot experience on iOS 26. The Screen Capture page in the Settings app lets you customize your screenshot experience in multiple ways. Here's how use can use the different features Apple has added in iOS 26. Turn on full-screen previews by default When you turn on the "Full-Screen Previews" toggle button on the Screen Capture page, iOS 26 will display screenshots in full view instead of showing a small thumbnail in the bottom left corner that vanishes after a few seconds. The full-blown UI allows you to quickly edit or annotate a screenshot and share it with your friends across various apps. Alternatively, you can disable the toggle option and trigger this UI by tapping on the temporary thumbnail that shows up after taking the screenshot. Prevent CarPlay screenshots from flooding your device Apple has also changed (fixed) how screenshots work on CarPlay. Until now, screenshots on CarPlay have been an "it's a feature, not a bug" experience for the users. If your iPhone is connected to your car, an extra screenshot of the CarPlay interface is automatically captured when you press the buttons to take a screenshot on your phone. This functionality has been a pain point for many users, who have raised their voice on Apple's support forums. It seems the company heard them, and iOS 26 update brings a new toggle button in the picture, allowing you to have a manual way to enable or disable CarPlay screenshots. Use Visual Intelligence on screenshots iPhone's screenshot tool also supports the improved Visual Intelligence feature (works on iPhone 15 Pro or later). You can select objects in a screenshot and search for matching content in apps that offer integration with Visual Intelligence. These screenshot updates for iOS 26 are currently available for testing through the developer beta program on supported iPhone models. Apple is expected to release the first public beta of iOS 26 next month. If you plan to install the developer beta on your device, note that early builds may be unstable, and ensure that you back up your data in advance.
    • Looking around, it seems to indicate that an App is doing a download in the background.
    • Dopamine 3.0 Preview 39 by Razvan Serea Dopamine is an awesome free audio player which tries to make organizing and listening to music as simple and pretty as possible. Dopamine has been designed for Windows 7, Windows 8.x and Windows 10 and plays mp3, ogg vorbis, flac, wma and m4a/aac music formats quite well. The best part? It's created by long-time Neowin member, Raphaël Godart. If you’re looking for a music player to handle a large music collection, you should definitely give Dopamine a try. Dopamine 3.0 Preview 38 changelog: Added Added Ctrl-F shortcut to focus the search bar and ESC to clear it Added support for cover art images that have file name albumart.jpg, albumart.jpeg or albumart.png Added Turkish translation Changed Slightly improved the look of the mini player Improved scaling of text on Now Playing screen for smaller screens Order selections are now menus instead of toggle buttons Updated the Brazilian Portuguese translation Updated the Spanish translation Updated the Turkish translation Updated the Vietnamese translation Fixed Fixed an issue where a random song would start playing after the queue ended when using gapless playback. Fixed an issue where album lists were not refreshed after a search returned no results. Fixed an issue where progress was not reset to 0 when playback has finished Fixed an issue where a track that is longer than 4 minutes was scrobbled to Last.fm immediately after it started playing Fixed some sizing issues on the "Folders" screen Fixed search problems Download: Dopamine 3.0 Preview 39 | 98.4 MB (Open Source) View: Dopamine Home Page | Forum Discussion | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • Here's what to expect from Samsung's July Unpacked event by Hamid Ganji Samsung held its first Unpacked event of the year in January to unveil the Galaxy S25 series. However, the Korean OEM is also gearing up to hold its second 2025 Unpacked event sometime next month, focusing on its next-generation foldable phones and a new pair of Galaxy Watch wearables. Samsung might also give us a detailed look into the upcoming Project Moohan XR Headset and a rumored tri-folding smartphone that might launch later in the year or early 2026. One UI 8 is another topic Samsung might discuss at the event. Here's everything we expect Samsung to unveil at next month's Unpacked event. When the event kicks off To date, there have been conflicting rumors about when the Unpacked event will start. However, the latest hint comes from renowned tipster Evan Blass, who says the second 2025 Unpacked event begins on July 9, 10 AM EDT. Samsung has yet to confirm the alleged date, but it aligns with the company's unveiling of the Galaxy Z Fold 6 and Flip 6 on July 10 last year. New foldables are down the road Galaxy Z Fold 7 and Galaxy Z Flip 7 will take the stage at the upcoming Unpacked event. Both phones will offer upgrades compared to their predecessors, but the overall design is expected to remain relatively the same. Some sources have claimed Z Fold 7 would be Samsung's thinnest foldable ever, presumably 4.5mm thick when fully unfolded. Also, Z Fold 7 might get a bigger screen at 6.5 and 8.2-inch. Samsung might also switch to a 200MP camera in the lineup. Other rumored specs for Samsung's next-gen foldable include Snapdragon 8 Elite, a 4,400 mAh battery, and One UI 8 out of the box. Meanwhile, there are also some swirling rumors about a so-called Galaxy Z Flip 7 FE (Fan Edition) in the pipeline. The device could be a stripped-down version of Samsung's Galaxy Z Flip 7 with lower hardware specs and a cheaper price tag. A new pair of smartwatches Samsung will also reveal the Galaxy Watch 8 and a Classic variant at July's Unpacked event. As for design, we don't expect a radical makeover for the Galaxy Watch 8, but the Classic model could take most of its cues from Samsung's Galaxy Ultra watch models. Also, the Watch 8 Classic could bring back the rotating bezel on top. Both watches will come with One UI 8 Watch out of the box and Gemini AI to replace the Assistant. Gemini could help with health features and keep track of users' physical activities. Samsung was also rumored to be working on Blood glucose monitoring for next-gen smartwatches, and it might finally make its debut at the upcoming Unpacked event. More details about July's Unpacked event will surface in the days leading up to it. You can watch the live stream of the event through Samsung's website or the company's YouTube channel.
    • Can't they just allow people to hide the status bar (battery etc.) when Safari is open and to hide the bottom bar completely when scrolling for a true full-screen experience ? Samsung does that on its Samsung Internet Browser and it's a very good experience.
  • Recent Achievements

    • One Month Later
      serfegyed earned a badge
      One Month Later
    • Dedicated
      firey earned a badge
      Dedicated
    • Dedicated
      fettermanj earned a badge
      Dedicated
    • One Month Later
      SekTheFirst earned a badge
      One Month Later
    • First Post
      zayanhani earned a badge
      First Post
  • Popular Contributors

    1. 1
      +primortal
      636
    2. 2
      ATLien_0
      231
    3. 3
      Michael Scrip
      219
    4. 4
      Xenon
      145
    5. 5
      Steven P.
      141
  • Tell a friend

    Love Neowin? Tell a friend!