Skip to content

fix: add explicit err msgs for missing pckg extras during import#165

Merged
vdusek merged 8 commits into
masterfrom
err-msg-when-missing-extra
May 31, 2024
Merged

fix: add explicit err msgs for missing pckg extras during import#165
vdusek merged 8 commits into
masterfrom
err-msg-when-missing-extra

Conversation

@vdusek

@vdusek vdusek commented May 31, 2024

Copy link
Copy Markdown
Collaborator

Description

  • Add explicit error messages for missing package extras during import.
  • It applies to BrowserPool, BeautifulsoupCrawler, and PlaywrightCrawler.
  • Also until now the Playwright was mandatory for even BasicCrawler, fixing it.

Related issues

Testing

PlaywrightCrawler

ImportError                               Traceback (most recent call last)
Cell In[3], line 1
----> 1 from crawlee.playwright_crawler import PlaywrightCrawler

File ~/Projects/crawlee-py/src/crawlee/playwright_crawler/__init__.py:5
      3     from .types import PlaywrightCrawlingContext
      4 except ImportError as exc:
----> 5     raise ImportError(
      6         'To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".',
      7     ) from exc

ImportError: To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".

BeautifulsoupCrawler

ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 from crawlee.beautifulsoup_crawler import BeautifulSoupCrawler

File ~/Projects/crawlee-py/src/crawlee/beautifulsoup_crawler/__init__.py:5
      3     from .types import BeautifulSoupCrawlingContext
      4 except ImportError as exc:
----> 5     raise ImportError(
      6         'To use this module, you need to install the "beautifulsoup" extra. Run "pip install crawlee[beautifulsoup]".',
      7     ) from exc

ImportError: To use this module, you need to install the "beautifulsoup" extra. Run "pip install crawlee[beautifulsoup]".

BrowserPool

ImportError                               Traceback (most recent call last)
Cell In[2], line 1
----> 1 from crawlee.browsers import BrowserPool

File ~/Projects/crawlee-py/src/crawlee/browsers/__init__.py:5
      3     from .playwright_browser_plugin import PlaywrightBrowserPlugin
      4 except ImportError as exc:
----> 5     raise ImportError(
      6         'To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".',
      7     ) from exc

ImportError: To use this module, you need to install the "playwright" extra. Run "pip install crawlee[playwright]".

Checklist

  • Changes are described in the CHANGELOG.md
  • CI passed

@github-actions github-actions Bot added this to the 90th sprint - Tooling team milestone May 31, 2024
@github-actions github-actions Bot added the t-tooling Issues with this label are in the ownership of the tooling team. label May 31, 2024
@vdusek vdusek changed the title fix: better error message when missing package extra fix: add explicit err msgs for missing pckg extras during import May 31, 2024
@vdusek vdusek marked this pull request as ready for review May 31, 2024 10:36
@vdusek vdusek requested a review from janbuchar May 31, 2024 10:39
Comment thread src/crawlee/beautifulsoup_crawler/__init__.py Outdated
Comment thread src/crawlee/basic_crawler/basic_crawler.py
@vdusek vdusek requested a review from janbuchar May 31, 2024 12:07
@vdusek vdusek requested review from janbuchar and removed request for janbuchar May 31, 2024 14:05
Comment thread src/crawlee/browsers/browser_pool.py Outdated
@vdusek vdusek requested a review from janbuchar May 31, 2024 14:52
@vdusek vdusek merged commit 200ebfa into master May 31, 2024
@vdusek vdusek deleted the err-msg-when-missing-extra branch May 31, 2024 15:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

t-tooling Issues with this label are in the ownership of the tooling team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve error message when importing missing package extras

2 participants