Scala/Spark Developer, REMOTE

• Strong commercial experience developing production applications in Scala, including functional programming concepts (immutability, pattern matching, higher-order functions), Scala type system, collections, sbt, and testing frameworks such as ScalaTest or munit, with the ability to write clean, maintainable, production-quality code;
• Proven hands-on experience with Apache Spark, including Spark SQL, DataFrame/Dataset API, designing and maintaining ETL/ELT pipelines, performance optimization (partitioning, shuffle optimization, broadcast joins, caching), and troubleshooting using Spark UI;
• Experience working in production environments with large-scale data processing and distributed data pipelines;
• Strong understanding of software engineering best practices, code quality, performance optimization, and maintainability.

Nice to have

• Experience with Kafka and Spark Structured Streaming;
• Experience with Airflow or similar workflow orchestration tools;
• Experience with cloud platforms such as AWS (EMR, S3, Glue), Azure, GCP (Dataproc), or Databricks;
• Knowledge of the big data ecosystem, including Hadoop, Hive, Delta Lake, Parquet, and Avro;
• Experience with SQL, data modeling, data warehouses, and data lake architectures;
• Experience with Docker, Kubernetes, Git, and CI/CD pipelines;
• Familiarity with Scala libraries such as Cats, ZIO, Akka, or Pekko;
• Knowledge of Python or Java.