Job Overview

We are seeking a skilled and experienced Python Lead - Web Scraping to join our team. As a Python Lead - Web Scraping, you will be responsible for developing and maintaining web scraping solutions to gather data from various sources.

Qualifications and Skills

• 4 to 6 years of experience in web scraping using Python.
• Strong knowledge of web scraping frameworks such as BeautifulSoup and Scrapy.
• Proficiency in Python programming language.
• Experience with data extraction, transformation, and loading techniques.
• Familiarity with database systems and SQL.
• Good understanding of HTML, CSS, and XPath.
• Strong analytical and problem-solving skills.
• Excellent attention to detail and accuracy.
• Ability to work both independently and collaboratively in a team environment.
• Strong communication and interpersonal skills.
• Leadership skills to guide and mentor junior team members.

Roles and Responsibilities

• Develop and maintain web scraping solutions using Python.
• Design and implement data extraction, transformation, and loading strategies.
• Collaborate with cross-functional teams to understand data requirements and deliver high-quality solutions.
• Optimize web scraping processes and workflows to improve efficiency and performance.
• Framework for expectations of each data collection type (DS, SS, etc.)
• Stay up-to-date with the latest web scraping tools, techniques, and best practices.
• Train and mentor junior team members on web scraping techniques and best practices.
• Close coordination with QA team
• Prioritization of what to work on
• Team Management & Hiring
• Cost monitoring
• Oversight of requests, requirements, and inputs from backend team as well as clients
• Oversight of engagement with vendors (Bright Data, Oxylabs, etc.)
• Framework for how to raise tickets and what should be included in tickets
• Escalation (control over when and how to go through sales team on the vendor side)