About the role:

We are looking for model safety engineers in the text and image modality who are passionate about generative models and creative applications of AI and at the same time understand the risks they can pose. In particular, we are looking for individuals with experience in red teaming as well as defenses of attacks when it comes to malicious prompts for text-to-image and language models. The ideal candidate will have a deep understanding of natural language processing (NLP) techniques and a track record of developing and implementing effective strategies to mitigate risks associated with text-to-image and language models. They should be proactive in identifying potential vulnerabilities and possess a hacker mindset to anticipate and counteract malicious inputs. Additionally, a strong ethical compass and a commitment to the responsible use of AI technology are essential attributes we seek in the ideal Model Safety Engineer.

Responsibilities:

• Conduct red teaming exercises to identify vulnerabilities and potential misuse of AI models.
• Develop and implement defense strategies to protect against harmful inputs and outputs.
• Evaluate and deploy guardrails to ensure the responsible and ethical use of our models.
• Collaborate with cross-functional teams to integrate safety measures into model development pipelines.
• Stay up-to-date with the latest research and advancements in AI safety to continuously improve our safeguards.

Qualifications:

• 3+ years working experience in AI safety, ethics, or security, with a focus on text-to-image and language models.
• Strong programming skills in Python and experience with relevant libraries and frameworks.
• Knowledge of AI/ML model architectures and the ability to assess their vulnerabilities.
• Familiarity with red teaming methodologies and ethical hacking practices of AI models.
• Excellent communication skills to convey complex technical concepts effectively.
• Experience with cloud platforms like Amazon AWS or Google Cloud is a plus.
• Commitment to the ethical use of AI and a dedication to ensuring AI technology benefits society.
• Ability to communicate machine learning concepts and results effectively through writing and visualization.
• Ability to work in a fast-paced, remote, and collaborative startup environment.