Data Engineer jobs

Software Engineer

SpellbrushSan Francisco, CA +1 placeFull-timeOn-site

Pay not listed

Key facts

Where
San Francisco, CA · Tokyo, Japan · on-site
Experience
3+ years
US visa
Sponsors US visas

2 things to check before you apply.

On Spellbrush’s own careers site (Ashby), checked open today.

  • To check: Posted by the employer 2 years agoLong-open jobs are sometimes already filled. Ask when they plan to hire.
  • To check: No pay range listed, though one is likely required in CAAsk for the pay range before you interview. The law here likely requires one.
  • Checked: On the company's own careers siteAshby board
  • Checked: Checked open todayAshby board
  • Checked: No reposts foundAshby board
  • Trust score 55 / 100 · Check details
See all 6 checksShow fewer
How we check

What you’ll do

  • Define processes and infrastructure to transform data and make it available across the company.
  • Partner with AI researchers to understand their needs and design, build, and monitor scalable data pipelines.
  • Build pipelines and tools for data such as embeddings, video, and bounding boxes.

Summarised from the posting by AI and can be wrong. The full description is the source.

Requirements

Experience

  • 3+ years as a software engineer
  • Experience with distributed systems
  • Previous administration of a big data toolset such as Ray, Spark, or Airflow
  • Hands-on experience shipping scalable data solutions in the cloud across multiple data stores
  • Hands-on experience designing and building highly scalable and reliable data pipelines (preferred)
  • Experience building large-scale big data pipelines (preferred)
  • Comfort working on small, fast-paced teams (preferred)

Skills

  • A wide generalist skillset
  • Ability to pull data from varied sources, including writing complex SQL queries and coordinating large data transfers
  • Comfort with scripting in TypeScript, Python, or Bash

Other requirements

  • Ability to work directly with the research team to gather and synthesize needs, design and implement solutions, and communicate deliverables, timelines, and tradeoffs
  • Interest in learning from and interfacing with AI researchers (preferred)
  • Being a fan of anime or the anime aesthetic (preferred)

Your match

Sign in and add your resume to see which of this job's requirements it shows.

Sign in to check your match

Saved to your account, encrypted. The AI that reads it doesn't keep it or train on it.

Benefits

Other
Visa sponsorships are available.

Full job description

From Spellbrush’s careers site

[https://app.ashbyhq.com/api/images/user-content/89c1442c-bd9c-46be-a910-20b49b5d9ffc/20ffb2db-1beb-429d-894f-ebb50a85bdb2/image.png]

GENERALIST SOFTWARE ENGINEER

ABOUT US

Here at Spellbrush, we're passionate about making a good anime game.

We also happen to be the world's leading generative AI studio — we're the team behind niji・journey https://nijijourney.com/.

We are currently investigating how AI can be used to help human artists perform masterpieces in the most complex medium of our times: videogames.

Our games are characterized by a no-compromise approach to well-balanced gameplay married to a truthful love of visual arts.

If you love turn-based tactics games, please consider applying!

Show full descriptionShow less

ABOUT THE ROLE

As Spellbrush continues to grow rapidly, we're seeking talented generalist engineers to join our team and help us build advanced data pipelines, foundational datasets, as well as the tools and infrastructure that support them. If the thought of running a batch job across ten-thousand GPUs to ingest petabytes worth of anime image and videos excites you, this is the job for you!

WHAT YOU'LL ACHIEVE

You'll define the processes and infrastructure to transform and make data readily available across the company, whether it be embeddings, video, bounding boxes, or more.

You'll partner with our world class AI researchers to understand their needs, and then design, build and monitor pipelines that meet today's requirements but can gracefully scale with our growing data size.

SKILLS YOU'LL NEED TO BRING

You are a self-starter and can work directly with the research team to gather and synthesize high-impact needs from teammates, designing and implementing the appropriate technical solutions, and effectively communicating about deliverables, timelines and tradeoffs.

You've spent 3+ years as software engineer, with a wide generalist skillset.

You’ve worked with distributed systems, and previously administered a big data toolset (Ray, Spark, Airflow, etc.).

You have hands-on experience shipping scalable data solutions in the cloud (e.g AWS, GCP, Azure, etc.), across multiple data stores (e.g Snowflake, Redshift, Hive, SQL/NoSQL, etc.).

You’re comfortable pulling data from whatever source it may be in, whether that involves writing a complex SQL query to pull a billion rows out of a Bigquery Database, or coordinating the transfer of 10+ PB of data across two S3 regions.

You are highly comfortable with scripting, whether in Typescript, Python, or even Bash when the need arises.

Most importantly, you know when to pull in the complex overengineered tools, and when to instead rely on duct tape.

YOU MAY BE A GOOD FIT IF:

YOU HAVE HANDS-ON EXPERIENCE IN DESIGNING AND BUILDING HIGHLY SCALABLE AND RELIABLE DATA PIPELINES

Building data pipelines for AI presents itself with unique challenges. If you’ve previously built large scale big data pipelines, you’d be a good fit.

YOU’RE INTERESTED IN LEARNING AND INTERFACING WITH AI RESEARCHERS

You’ll interface with some of the best AI researchers in the world, meaning that you’ll get to learn from the best. Your code will frequently need to ingest values and embeddings from our state-of-the-art models, and we’ll help you get up to speed here.

YOU LOVE ANIME AND THE ANIME AESTHETIC

As a member of our team, you'll have the opportunity to push the boundaries of what's possible in the anime and video game industry. Ideally you're a fan of the genre.

YOU'RE COMFORTABLE WORKING ON SMALL, FAST-PACED TEAMS

Our engineering team is small but mighty. You'll be working directly with some of the best AI researchers and engineers in the world to integrate their state-of-the-art models into your beautiful interfaces.

We also believe in the unmatched speed of in-person teams, and prefer on-site collaboration in either our primary research office in Tokyo (downtown Akihabara) or San Francisco. Visa sponsorships are available.

[https://app.ashbyhq.com/api/images/user-content/89c1442c-bd9c-46be-a910-20b49b5d9ffc/9615e43d-d61e-456f-9117-ce2f2537a42f/image.png]

Something wrong with this job? Report it · This flag looks wrong

Similar jobs nearby

Data Engineer II

Knit Health · San Francisco, CA

$150k–$175k/yrFull-timeHybrid

StrongLast seen open yesterdayPosted 23 days ago

Senior Data Engineer

BuildOps · Toronto, CA +2 places

$120k–$197k/yrHybrid

StrongChecked open todayPosted 11 days ago

See more like this