Senior Spark Developer- (Hybrid)
Citi
We are seeking a highly skilled and experienced Senior Spark Developer to join our team. This role demands a deep understanding of Apache Spark and its ecosystem, along with a strong foundation in Java/J2EE application development. You will be responsible for designing, developing, and optimizing Spark-based solutions to process large volumes of data, collaborating with various teams to deliver high-performance and scalable data applications. This is a challenging and rewarding opportunity to contribute to a cutting-edge data engineering team.
Responsibilities:
- Design, develop, and implement complex Spark applications for processing large datasets, leveraging your expertise in distributed computing and data manipulation techniques.
- Partner with management teams to integrate Spark-based solutions into existing data infrastructure, ensuring seamless data flow and optimal performance.
- Resolve critical data processing challenges by evaluating complex business processes, system processes, and industry standards, applying your deep understanding of Spark's capabilities.
- Provide technical leadership and mentorship to other engineers, guiding them on Spark best practices and promoting efficient development strategies.
- Develop and maintain coding standards, testing procedures, and debugging techniques specifically for Spark applications.
- Contribute to the overall data architecture, ensuring that Spark solutions align with business goals and integrate effectively with other data technologies.
- Appropriately assess risk when making technical decisions, considering factors such as data security, performance, and scalability.
- Drive compliance with applicable laws, rules, and regulations, adhering to company policies and demonstrating sound ethical judgment.
Qualifications:
- 6+ years of application development experience, with a significant focus on Java/J2EE and Apache Spark.
- Strong command of Object-Oriented Programming (OOP) principles and design patterns in Java.
- Excellent understanding of data structures and algorithms, and their application in Spark development.
- Proven experience developing complex Java/J2EE applications, encompassing all areas of the technology stack.
- Thorough knowledge and hands-on experience with Hadoop, MapReduce, YARN, Sqoop, Pig, Hue, Unix, Impala, and Cassandra on Mesos.
- Extensive experience designing, developing, and deploying Spark applications in a production environment, handling large volumes of data.
- Deep understanding of Spark architecture, including Spark SQL, Spark Streaming, RDDs, and DataFrames.
- Experience with ETL and BI tools is a plus.
- Experience working in an agile environment using Scrum methodologies.
- Expertise in designing and optimizing software solutions for performance and stability, particularly in the context of Spark.
- Expertise in troubleshooting and problem-solving complex technical issues.
- Experience with test-driven development (TDD).
- Cloudera certification (CCDH) is an added advantage.
Preferred Qualifications:
- Experience with cloud-based Spark platforms (e.g., Databricks, AWS EMR, Azure HDInsight).
- Experience with other big data technologies (e.g., Kafka, Hive, HBase).
- Experience with data visualization tools and techniques.
Education:
- Bachelor's degree in Computer Science, Engineering, or a related field.
This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.
------------------------------------------------------
Job Family Group:
Technology------------------------------------------------------
Job Family:
Applications Development------------------------------------------------------
Time Type:
Full time------------------------------------------------------
Most Relevant Skills
Please see the requirements listed above.------------------------------------------------------
Other Relevant Skills
For complementary skills, please see above and/or contact the recruiter.------------------------------------------------------
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.
View Citi’s EEO Policy Statement and the Know Your Rights poster.