Data Platform Engineer

**About the company:** VISASQ Meguro-ku, Tokyo VisasQ operates a global knowledge platform connecting organizations with more than 800,000 experts across 190 countries. Serving over 2,000 clients worldwide, the company helps businesses access first-hand insights through interviews, surveys, and advisory work. Read more **Responsibilities:** Lead the entire technical lifecycle of our data foundation—from architectural design supporting search, analytics, and compliance, to entity resolution (deduplication), data cleansing, and building mechanisms for continuous data quality assurance. Data Platform Architectural Design: Design data models across multiple products and multi-cloud environments (Azure/GCP) to ensure reliable data supply for AI products, search engines, analytics, and compliance checks. Define Single Source of Truth (SSOT) strategies and integrate confidence scoring into data models based on empirical findings. Entity Resolution & Data Cleansing Pipeline: Design and implement multi-stage entity resolution pipelines combining deterministic matching, scoring models, and human-in-the-loop review workflows. Build robust data cleansing processes that enforce idempotency and state rollback capabilities to maintain high data integrity over time. Establish monitoring and anomaly detection mechanisms to ensure stable pipeline operations. Data Quality Research & Problem Solving: Investigate data quality issues, identify root causes within data/codebases, and formulate impactful technical solutions based on business metrics. Technical Decision-Making & Stakeholder Alignment: Document architectural designs, research findings, and Architectural Decision Records (ADRs). Drive consensus with the CTO, local product teams, and global engineering leaders. Requirements Data Engineering & DWH Experience: Proven track record of designing, building, and operating production-grade data pipelines and data warehouse architecture. Relational Database Expertise: Deep hands-on experience with SQL execution plan analysis, index optimization, and high-volume batch processing. Root-Cause Data Investigation: Experience cross-analyzing code logic and raw data to resolve data inconsistencies, pipeline outages, and performance bottlenecks. Technical Documentation & Consensus Building: Strong ability to document architectural choices (design docs, research reports, ADRs) and align technical direction with stakeholders. Backend Development: Hands-on experience developing backend applications in modern programming languages. Language Proficiency: Native or bilingual level Japanese proficiency (essential for analyzing complex local data schemas and collaborating with Japanese domain teams). Nice to haves While not specifically required, tell us if you have any of the following. AI/Search Data Pipelines: Experience providing data foundations for AI/LLM products, RAG systems, or enterprise search platforms. Master Data Management (MDM): Experience with record linkage, entity resolution, deduplication algorithms, and continuous data governance. Master Data Migration: Experience leading multi-source database consolidation, unified schema design, ID mapping, and zero-downtime strangler-pattern migrations. Distributed System Data Syncing: Debugging and development experience in hybrid environments with event-driven and batch-oriented data syncs. High-Precision Domains: Engineering experience in domains requiring extreme data precision (e.g., finance, healthcare, authentication, compliance). Cloud & Data Stack: Hands-on experience with SQL Server, Azure (AKS, Data Factory), or GCP (BigQuery, Cloud SQL). Compensation ¥7,000,000 ~ ¥12,000,000 annually. **Requirements:** Data Engineering & DWH Experience: Proven track record of designing, building, and operating production-grade data pipelines and data warehouse architecture. Relational Database Expertise: Deep hands-on experience with SQL execution plan analysis, index optimization, and high-volume batch processing. Root-Cause Data Investigation: Experience cross-analyzing code logic and raw data to resolve data inconsistencies, pipeline outages, and performance bottlenecks. Technical Documentation & Consensus Building: Strong ability to document architectural choices (design docs, research reports, ADRs) and align technical direction with stakeholders. Backend Development: Hands-on experience developing backend applications in modern programming languages. Language Proficiency: Native or bilingual level Japanese proficiency (essential for analyzing complex local data schemas and collaborating with Japanese domain teams). **Nice to have:** While not specifically required, tell us if you have any of the following. AI/Search Data Pipelines: Experience providing data foundations for AI/LLM products, RAG systems, or enterprise search platforms. Master Data Management (MDM): Experience with record linkage, entity resolution, deduplication algorithms, and continuous data governance. Master Data Migration: Experience leading multi-source database consolidation, unified schema design, ID mapping, and zero-downtime strangler-pattern migrations. Distributed System Data Syncing: Debugging and development experience in hybrid environments with event-driven and batch-oriented data syncs. High-Precision Domains: Engineering experience in domains requiring extreme data precision (e.g., finance, healthcare, authentication, compliance). Cloud & Data Stack: Hands-on experience with SQL Server, Azure (AKS, Data Factory), or GCP (BigQuery, Cloud SQL). **Compensation:** ¥7,000,000 ~ ¥12,000,000 annually.