Deskripsi Pekerjaan
Informasi lengkap tentang posisi dan persyaratan
Ringkasan Yukerja
Lowongan AI QA Engineer (Lippo Malls Head Office) di Lippo Malls Indonesia kami kurasi dari JobStreet (kategori Teknologi & IT). Perhatikan lokasi kerja (West Jakarta, Jakarta) sebelum melamar. Yukerja.com bukan pemberi kerja — lamaran diproses di situs sumber resmi.
Job Description
Monitor and evaluate GenAI performance metrics through AI dashboards.
Identify anomalies, accuracy gaps, or biases in system responses.
Conduct regular reviews of model quality and user experience outcomes.
Perform Human-in-the-Loop (HITL) testing by recommending test scenarios and use cases to validate GenAI responses.
Collaborate with cross-functional teams to ensure human evaluation is effectively integrated into the AI development process.
Analyze GenAI outputs and recommend reinforcement learning strategies.
Provide actionable recommendations for prompt tuning, model refinement, and Retrieval-Augmented Generation (RAG) optimization.
Review sentiment analysis results from user feedback (positive, negative, and neutral).
Collaborate with cross-functional teams to generate insights that support continuous improvement of GenAI systems.
Ensure QA processes comply with company standards, AI ethics guidelines, and regulatory requirements.
Prepare documentation for QA testing protocols, evaluation results, and improvement
Requirements
Bachelor's degree in Computer Science, Data Science, Engineering, or a related field (Master's degree is a plus).
Experience evaluating GenAI systems using frameworks such as OpenAI Evals, RAGAS, and custom evaluation/scoring methods.
Strong understanding of cloud-based AI platforms (e.g., OpenAI, Gemini, Llama) and Generative AI concepts, including RAG pipelines, embeddings, and LLM evaluation.
Proficient in query analysis, data validation, and AI evaluation metrics such as accuracy, precision, recall, BLEU, and ROUGE.
Knowledge of or experience with feedback loops, reinforcement learning, and basic Python programming for AI quality assurance.
Detail-oriented with strong problem-solving skills and the ability to design and execute AI system testing and evaluation plans.
Experienced in sentiment and text analysis of user feedback, with a background in AI Quality Assurance (QA) or AI Ethics & Governance projects.