So I am a brand new user of Selenium in Python. I am really enthusiastic about using this framework, but I’m encountering some issues with certificate errors. When I Google for this issue, I noticed several users in the past year having similar issues but nobody really providing a workable solution.

I’m essentially testing out Selenium by trying out the first script in the documentation before cutting my teeth on scraping my actual target website which I plan to do after everything is smooth with the first script that acts as my baseline to ensure that everything in my development environment is setup right:

https://www.selenium.dev/documentation/webdriver/getting_started/first_script/

Note that the official documentation never actually tells me to install the Selenium web driver. It just tells me to install Selenium on my Python virtual environment which I did using “pip install selenium”.

However at the time of reviewing the above documentation I was aware that a separate web driver for the web browser of choice was needed too. In my case I use Chrome. I decided I would first just follow the official documentation and see what would happen.

I pasted in the code verbatim from the Selenium github (https://github.com/SeleniumHQ/seleniumhq.github.io/blob/trunk/examples/python/tests/getting_started/first_script.py#L4) into first_script.py and then ran it in my python interpreter.

To my surprise it did spawn a browser briefly and then closed. But it’s throwing up several certificate errors:

[2944:20700:1219/211513.232:ERROR:cert_issuer_source_aia.cc(34)] Error parsing cert retrieved from AIA (as DER):
ERROR: Couldn't read tbsCertificate as SEQUENCE
ERROR: Failed parsing Certificate

Upon further inspection the script does appear to submit the form and then scrape the form results page as it correctly scripts “Received!” and stores it as a string in the “text” variable.

However the certificate errors are a huge problem I need resolved. My normal Chrome installation doesn’t throw up any certificate errors on https://www.selenium.dev/selenium/web/web-form.html which the above Selenium script submits and then scrapes the result. The web form url has a perfectly valid lets encrypt SSL certificate.

Remember I mentioned that I didn’t install the web driver since it wasn’t mentioned in the official document? I decided to rectify that since I figured maybe that could be the problem.

Since I am using Chrome 120.0.6099.110 I went to https://sites.google.com/chromium.org/driver/downloads?authuser=0 which instructed me to go here: https://googlechromelabs.github.io/chrome-for-testing/

I grabbed chromedriver.exe that matches my web browser and dropped it into my project directory. Then I modified first_script.py to ensure that it calls that specific web driver also commented out the original code:

#driver = webdriver.Chrome()
service = Service('C:\\py311seleniumbot\\chromedriver.exe')
driver = webdriver.Chrome(service=service)

Well it’s using the new chromedriver.exe alright but I’m still getting the same cert error.

I’m at a loss on how to fix this. I need there to be a proper https connection to the remote website. I don’t want to sweep this under the rug by using the “–ignore-certificate-errors” flag as that doesn’t address the security concerns of a production environment.

Is there a kind soul in this wonderful Selenium community that can help a budding selenium enthusiast?

If I can get this pesky certificate issue resolved, I’m going to be well on my way to becoming a Selenium fanatic and very excited to start automating with this beauty.

  • seleniumstudent@lemmy.worldOP
    link
    fedilink
    arrow-up
    1
    ·
    11 months ago

    Some people on stackoverflow suggest adding:

    options = webdriver.ChromeOptions()
    options.add_experimental_option('excludeSwitches', ['enable-logging'])
    

    Doesn’t this just sweep the problem under the rug and result in the Selenium controlled browser connecting to the remote website in an insecure manner? Proper https security needs to be happening.

    PLEASE can someone give me a clear answer on this? It’s really sad how this community died after it was moved off reddit.