Amazon Web Services / Engineering case study
Making model serving easier to diagnose.
Mapping service dependencies, monitoring infrastructure health, and making audit tooling easier to operate.
The work, at a glance
On AWS’s SageMaker / Bedrock infrastructure team, I built tools that helped engineers understand the systems behind a model endpoint. My work connected dependency visibility, live health monitoring, and containerized audit infrastructure.
Topology & health
A Go CLI for dependency mapping, paired with real-time infrastructure health monitoring.
Audit infrastructure
Containerization tooling built with AWS Copilot for the team’s audit systems.
- Bedrock endpoints in the serving environment
- 1M+
- Reduction in on-call response time
- ~70%
- Audit queries supported per week
- 10K+
01 / Context
An endpoint is only part of the system.
Model serving depends on more than the endpoint a customer calls. The SageMaker infrastructure powering Bedrock spans hardware, software, and downstream AWS services. When something failed, engineers needed to understand which of those dependencies was involved.
My primary project addressed that observability gap. The environment covered more than a million endpoints, and the existing diagnosis process offered limited visibility into dependent services. A useful operational tool had to make those relationships easy to inspect.
02 / Service topology
Start with the endpoint.
I built a Go-based CLI that mapped model endpoints to their downstream AWS service dependencies. The resulting topology gave on-call engineers a concrete starting point for root-cause analysis.
The important output was the relationship between an endpoint and the services it relied on. Exposing that relationship made it easier to move from a symptom at the serving layer to the infrastructure that needed investigation.
Make the serving system inspectable
03 / Monitoring
Pair the map with current conditions.
I also designed a real-time system health monitoring framework to give the team live visibility into infrastructure health. Proactive issue detection helped reduce on-call response time by approximately 70%.
Topology and health information answer complementary questions: which services are involved, and how are those services behaving now? Together, they made the serving environment easier to reason about during an incident.
04 / Operational tooling
Make the audit infrastructure repeatable.
I built containerization tooling using AWS Copilot, modernizing deployment of the audit infrastructure. The resulting setup supported more than 10,000 audit queries per week.
Containerized audit infrastructure
The CLI, health framework, and audit tools became part of the team’s production and on-call workflows. This made operational usability a core requirement of the work: engineers needed tools they could use while investigating real issues.
05 / Reflection
Reliability includes the tools around the service.
This was my first big-tech internship. Working with my mentors Jonathan and Javion, and my manager Qiyun, showed me how observability and operational rigor fit into production ML infrastructure.
I learned to design tooling around production constraints and to treat the on-call engineer’s workflow as part of the system. A service is easier to operate when the people responsible for it can understand its dependencies and current state.
Outside work, Seattle and Bellevue meant hiking at Lake Serene, late-night poker with other interns, and pickleball and boba after work.
