Job Description

Help evaluate AI assistants in realistic analytical workflows using cloud data warehouses. This remote contract role applies your data expertise to improve how AI systems analyze, reason about, and report on real-world data. Prior AI experience is not required. Key Responsibilities

• Complete structured evaluations that reflect common analytics work, including anomaly investigations, KPI reporting, and data refreshes, using AI-enabled tools connected to cloud data warehouses.
• Validate AI-generated figures against source data by writing and running SQL queries.
• Assess the accuracy of data joins, filters, and time windows used in analytical outputs.
• Maintain, reset, and manage seeded Snowflake datasets, ensuring data integrity and accurate answer states for test scenarios.
• Manage warehouse roles, permissions, and access controls to support secure, repeatable evaluation environments.
• Configure and document connectivity and authentication across analytics and SaaS platforms.
• Investigate and document new or undocumented product behavior encountered during workflows.
• Join calibration sessions with peers to support consistent, rigorous scoring and grading.

• At least 3 years of hands-on experience as a data analyst or analytics engineer.
• Advanced SQL skills and Snowflake expertise, including warehouses, access control, and query history.
• Experience auditing and reconciling reported metrics against raw data, with the ability to identify subtle aggregation or logic errors.
• Experience using connectors across Claude and ChatGPT.
• Familiarity with business and finance analytics, including KPI definitions and reporting for leadership or external stakeholders.
• Experience administering database access, roles, grants, and authentication or security protocols. Familiarity with OAuth or security integrations is a plus.
• Working knowledge of Slack, Google Workspace, or Microsoft 365 for sharing analytical outcomes.
• Experience using AI assistants for analytics and critically evaluating AI-generated SQL and outputs.
• Background in rubric-based evaluation, QA, or data labeling, along with strong written and verbal communication skills.

• Remote contract engagement.

• $50 to $60 per hour.

Job Tags

Similar Jobs