BRIEF DESCRIPTION:-
The Python/Spark/AI Developer role involves advancing data platforms and developing scalable AI-enabled solutions using Python, Spark, and lakehouse technologies. The main responsibilities include creating distributed data pipelines, optimizing Spark SQL and DataFrame APIs, and developing REST APIs. Key requirements include a bachelor's degree in Computer Science, over four years of Python development experience, and proficiency in Apache Spark/PySpark and Delta Lake.
Job Title: Remote Python/Spark/AI Developer Company: MSP Staffing (PTY) LTD
Salary: R62 500.00 - R87 500.00 Per Month Job Type: Permanent Sectors: IT Reference: Python2026
Vacancy Details
Employer: MSP Staffing (PTY) LTD
Our client is looking for a skilled Python/Spark/AI Developer to join a forward-thinking team focused on advancing data platforms and creating scalable AI-enabled solutions. This position is perfect for a developer passionate about distributed data, clean and robust Python code, APIs, and contemporary lakehouse technologies.
Key Requirements
Bachelor's degree in Computer Science, Engineering, or equivalent experience.
4+ years of professional Python development experience.
2+ years of production experience developing Spark or similar distributed data pipelines.
Proficient in Python 3.12, with a good understanding of type hints, Pydantic, and modern packaging practices.
In-depth knowledge of Apache Spark/PySpark, including Spark SQL, DataFrame APIs, partitioning, and performance optimisation.
Experience with Delta Lake or other technologies like Iceberg or Hudi.
Strong SQL skills and the ability to convert legacy T-SQL logic into Spark SQL.
Practical experience with pytest and automated testing.
Experience developing REST APIs with FastAPI or equivalent technologies.
Familiarity with Docker-based development environments.
Strong Git and CI/CD practices.
Advantageous
Experience integrating LLMs and AI services.
Knowledge of data privacy, PII handling, POPIA/GDPR concepts, and data governance.
Exposure to Hive Metastore, Trino, or Apache Ranger.
Azure data platform experience, including ADLS Gen2, Synapse, or ADF.
Experience with observability tools and structured logging.
Previous data migration and pipeline parity experience.
Databricks or equivalent Spark certification.
Interested candidates meeting these qualifications should send their CV to [Email hidden - Login to reveal]. You may also reach out to the IT team at XXX-XXXX or visit our website at https://mspstaffingza.co/.
NOTE: Include the reference number in the subject line when responding to the advert. We will only communicate with shortlisted candidates. If you do not hear from us within three days, please consider your application unsuccessful.
APPLY FROM SOURCE
Interested and qualified? Make sure you are logged in or register an account if you dont have one then click to goto Python/Spark/AI Developer – Remote application page to apply.
APPLICATION INFORMATION
-
Make sure you are logged in and your email is verified (Register if you dont have an account).
-
Click the Apply Now button above.
-
Your default email application or application link will open.
-
Attach your Application letter, updated Curriculum Vitae (CV) and any supporting documents required.
-
Use a professional subject line such as Application for "Indicate Position"
-
Send your application before the closing date.
Note: Only shortlisted candidates are contacted.
