Senior Software Engineer - Purview China Team

Key Skills

Amazon Simple Storage Service (S3), Apache Airflow, Apache Cassandra, Atlassian Suite, AWS, Azure, Azure CosmosDB, C# Programming, Elasticsearch, GoLang, Google Cloud Platform (GCP), HBase, Java Programming, Kubernetes-K8s, MongoDB, MySQL, Natural Language Processing (NLP), Oracle, REST API, SPARK Programming, SQL

Job Description

We are the Microsoft Purview team part of Microsoft Security organization. Microsoft Purview team presents interesting challenges in technologies such as data governance, data integration, big-data, cloud, storage and open-source. We build reliable, highly scalable and highly performing distributed systems for data governance on Azure. We work with various open-source technologies like Atlas, Kubernetes, Spark, GraphDB, Elastic Search, Egeria, Flow Engine, etc., and make contributions to these technologies. This is a once in a lifetime opportunity to be part of a very agile team, take on hard distributed system problems and ship mission-critical features at a rapid pace. Microsoft puts customers in control of their personal data and backs this commitment with continuous investment in governance infrastructure. We need collaborative developers who can think big, deliver on those big challenges, and along the way, change the world. We’re looking for engineers to build them from the ground up.

• Autonomy to drive major feature areas.
• Cutting edge technologies (Spark, Kubernetes, Atlas, Egeria, Elastic Search, Flow Engine, GraphDB, NLP, connector framework, Scanning, Airflow).
• Collaborative, supportive culture
• Faster domain knowledge and career growth due to high visibility and high business impact of the service.

Microsoft Purview China team has openings ranging from entry to Principal. We are seeking top talents with passion for data catalog, big data, data discovery and data governance. You are expected to learn all kinds of industrial data stores, e.g. sap, databricks, mongodb, salesforce, snowflake, etc. and build a huge set of metadata connector to retrieve metadata and classification from those stores in an extensible approach. You will learn cutting edge big data services, like Atlas, lineage, Spark, graph and search engine. You also will learn deeper into kubernetes world, to build highly reliable, scalable and elastic cloud service, which is operated at all Microsoft Azure data centers and other Cloud Providers. You will have opportunities to reach out to customers to understand / solve a real customer pain points.

Qualifications

Required Qualifications:

• 7+ years of professional software development experience.
• Fluent in one or more programming language like Java, Golang and C#.
• Solid data structure knowledge and familiar with common algorithms.
• Good knowledge of OO design and basic understanding with functional programming concepts
• Experience on designing & building large scale cloud services and software systems.

Preferred Qualifications:

• BS/MS in Computer Science, Mathematics/Physics, or Engineering, or equivalent experience.
• Familiar with container related technologies (Docker, Kubernetes).

• Experience on customizing Kubernetes like developing operators and define CRDs.
• Experience with web service development and familiar with related technologies (SpringCloud, REST).
• Experience in data governance areas like data catalog, lineage, discovery and data policy, and familiar with related technologies like Apache Atlas, Egeria and OpenLineage.
• Solid knowledge on distributed system.
• Experience on designing and building large web services and familiar with related technologies (gRPC, & REST).
• Familiar with cloud platform like Azure, AWS, GCP, AliCloud and etc.
• Familiar with popular data store (relational, document, wide column, key-value, etc) like MySQL, Oracle, SQLServer, MongoDB, CosmosDB, Redis, Cassandra, Hbase, S3, Azure Storage, etc.
• Familiar with big data technologies like Spark, Hadoop, Flink and Kafka.
• Familiar with full text search technologies (Apache Lucene, Elastic Search & Apache Solr).
• Familiar with graph database (Neo4j, Gremlin & JanusGraph, etc)
• Familiar with machine learning and NLP.
• Familiar with workflow related technologies (Apache Airflow, etc).
• Familiar with service mesh technologies (Istio).
• Familiar with modern security models like OAuth and token based authentication and authorization.
• Experience building and shipping production grade software or services.
• Experience using agile methodologies or test-driven development (TDD).
• Great curiosity and willingness to question.
• High enthusiasm, integrity, ingenuity, results-orientation, self-motivation, and resourcefulness in a fast-paced competitive environment.
• Have a deep desire to work collaboratively, solve problems with groups, find win/win solutions and celebrate successes.
• Get excited by the challenge of hard technical problems.
• Solve problems by always leading with deep passion and empathy for customers.

Responsibilities

We are looking for a great software engineer with experiences in backend services and data pipelines to build the next generation of Microsoft's data governance services.

• Design and develop large scale, elastic distributed cloud systems.
• Deploy and operate services in production.
• Work with customers to resolve their issues and gather requirements for new features.
• Work with Microsoft stakeholders in and out of the immediate team to make sure our code is compliant and secure, as much as it solved the customer problems.