University of Michigan / Research notes
Using code structure to guide generation.
An AST-aware transformer with constrained attention, supported by a contrastive-learning pipeline for dataset expansion.
The work, at a glance
I worked with Professor Victor Liu on self-operating computer networks. My research focused on code generation and training-data expansion: how a model uses program structure, and how to grow the dataset available to it.
Structure-aware modeling
An AST-aware transformer that uses constrained attention for code generation.
Training-data expansion
A contrastive-learning pipeline for automated dataset expansion.
- Reported code-generation accuracy
- 92%
- Increase in training samples
- +250%
01 / Context
Make program structure part of the model.
Within the self-operating networks research, I designed an AST-aware transformer using constrained attention mechanisms for code generation. An abstract syntax tree (AST) represents the structure of a program, giving this work a structural context for generation.
The model work and the dataset-expansion work addressed separate aspects of the learning system.
02 / Research components
Work on both representation and data.
I implemented a contrastive-learning pipeline for automated dataset expansion. This increased the number of training samples by 250%.
Structure and data, together
The diagram distinguishes the generation path from the training-data path. The constrained-attention transformer is the modeling contribution; expanding the corpus is the data contribution.
03 / Outcomes
Track model performance and dataset growth.
The work achieved 92% code-generation accuracy for the AST-aware transformer and a 250% increase in training samples from the expansion pipeline.