Skip to content

feat: add transform_request_function for enqueue_links#923

Merged
janbuchar merged 13 commits into
apify:masterfrom
Mantisus:transform-request-function
Jan 27, 2025
Merged

feat: add transform_request_function for enqueue_links#923
janbuchar merged 13 commits into
apify:masterfrom
Mantisus:transform-request-function

Conversation

@Mantisus

Copy link
Copy Markdown
Collaborator

Description

  • Add transform_request_function for enqueue_links
  • Add example in introduction section
  • Add tests

Issues

@Mantisus Mantisus self-assigned this Jan 20, 2025
janbuchar

This comment was marked as resolved.

@Mantisus

This comment was marked as resolved.

@Mantisus Mantisus requested a review from vdusek January 21, 2025 08:28
Comment thread src/crawlee/_request.py Outdated
@janbuchar

This comment was marked as resolved.

@B4nan

This comment was marked as resolved.

@janbuchar

This comment was marked as resolved.

@B4nan

This comment was marked as resolved.

Comment thread src/crawlee/_types.py Outdated
@vdusek

This comment was marked as resolved.

Comment thread docs/introduction/code/03_transform_request.py Outdated
@Pijukatel

This comment was marked as resolved.

@janbuchar

This comment was marked as resolved.

@B4nan

This comment was marked as resolved.

Comment thread src/crawlee/_request.py
Comment thread src/crawlee/crawlers/_abstract_http/_abstract_http_crawler.py Outdated

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

Comment thread docs/introduction/03_adding_more_urls.mdx Outdated
Comment thread src/crawlee/_request.py
Comment thread src/crawlee/_request.py Outdated
Comment thread src/crawlee/_types.py Outdated
Mantisus and others added 3 commits January 23, 2025 16:13
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>
Co-authored-by: Vlada Dusek <v.dusek96@gmail.com>

@vdusek vdusek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!

Comment thread src/crawlee/_types.py
@Mantisus Mantisus requested a review from janbuchar January 27, 2025 12:10

@janbuchar janbuchar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of nitpicks 🙂

Comment thread docs/introduction/code/03_transform_request.py Outdated
Comment thread src/crawlee/_types.py
Comment thread src/crawlee/crawlers/_abstract_http/_abstract_http_crawler.py Outdated
@janbuchar janbuchar self-requested a review January 27, 2025 14:09
@janbuchar janbuchar merged commit 6b15957 into apify:master Jan 27, 2025
@vdusek vdusek added this to the 107th sprint - Tooling team milestone Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Support for Specifying Headers in context.enqueue_links Method

5 participants