Web Module#

Web Scraping Capabilities#

  • HTML parsing and data extraction

  • JavaScript execution and dynamic content handling via webdriver

HTTP Request Handling#

  • Protocol Support:

    • HTTP/HTTPS request execution

    • Custom header specification and management

    • Authentication mechanism support via RFC9421

    • Proxy and routing configuration

  • Error Handling:

    • Network timeout and retry mechanisms

    • HTTP error code interpretation

    • Connection failure recovery

Security and Compliance#

  • Whitelist top-level domain filtering

  • Whitelist port filtering

  • URL pattern matching and validation

  • Execution time limits and timeouts