ItInsightsWorld General: How I Trained Qwen3-1.7B on a 16 GB GPU to Avoid e
A little while back, an RL project about flower drawing suggested a quick 'artistic' test for Qwen3-1.7B: can a model produce lipograms on a 16 GB GPU? Whole books such as Gadsby and La Disparition use this trick, but the model sees tokens, not letters. That also explains the old strawberry failure, where it could not count the r. This article records the masks, generated answers, SFT runs, and GRPO runs that moved usable replies from 0% with instruction only to 60.5% after training.
Headline before and after
| Prompt | Base model summary | Trained path summary |
|---|---|---|
| Trees poem | Generic leaves-sway verse | Gust, branch, shadow, quiet rhythm |
| Best way to study | Review, practice, sleep | Plan, companion, mock test |
| Green foods | Spinach, broccoli, green tea | Spinach, broccoli, asparagus |
| Dog time | More walking, training, cuddling | Walk, play, prioritize it |
| Dad joke | Skeleton joke | Clock joke |
Task rules and scoring
Prompts came from Dolly and No Robots, after filtering items that cannot be answered without e, such as Who painted the Mona Lisa? Each run used a system message saying no e or E. Answers were capped at 300 tokens, about 200 words. The usable denominator was 200: 50 prompts with 4 answers each.
GPT-6 Luna judged the last two checks, answering the prompt and normal English, from 1 to 5, and an answer needed at least a 4 on both. The first check was a literal rule: no e, no E, and no accented look-alikes such as é.
Why simple methods failed
Instruction alone failed. Qwen3-1.7B placed e in all 200 evaluation answers, so unmasked usable stayed 0%. Direct RL with a letter-only reward also failed within a couple of hours: the LoRA adapter learned to avoid e, but often by saying nothing useful.
A mask helped: for every token containing e, set its score to negative infinity. It also blocks E and non-ASCII variants, so é cannot appear. The mask does not teach e-free wording, so output can repeat or misspell. The baseline before training:
| Setup | Usable |
|---|---|
| Unmasked instruction only | 0% |
| Masked instruction and mask | 26% |
SFT with generated answers
Existing e-free prose did not transfer well. Gadsby is 50,000 words, about 69,000 tokens, and training on it scored about 26%, the masked baseline. The answers were sometimes shorter, repeated, or misspelled.
| Prompt | Masked | Gadsby-trained model |
|---|---|---|
| Best way to study | Practice topics, flashcards, mock quizzs coherence 3 responds 5 | Study for a t repeats coherence 1 responds 1 |
| Amazon review for a dress | Sis bridal day, good fit coherence 2 responds 4 | Amazon rviw drrss coherence 2 responds 2 |
| Trees poem | Soft slow wind cafts coherence 3 responds 5 | Tall oak so tall sway coherence 2 responds 3 |
| Plantain vs banana | Tubular fruit explanation coherence 5 responds 4 | Tropical root explanation coherence 5 responds 5 |
One reason is that novels teach prose, not prompt answers; another is size. I then made e-free answers to prompts: 180 suitable prompts, frontier models, then correction loops because even frontier models needed word replacements; Qwen3-1.7B did not benefit from that loop. I trained adapters on 15 to 180 examples:
| Training examples | 0 | 15 | 30 | 45 | 90 | 135 | 180 |
|---|---|---|---|---|---|---|---|
| Usable under mask | 26% | 10% | 20.5% | 22.5% | 31.5% | 34% | 35% |
The best SFT adapter reached 35%, 9 points above masked. It was not just text volume: SFT used full chats up to 3,072 tokens, about 117,000 tokens total, versus 69,000 tokens for Gadsby. The 90-example run, the closest size match, still beat Gadsby. Gains flattened: doubling from 90 to 180 added 3.5 points. The 15 examples run made longer and worse answers. When a key word contains e, the adapter may switch topics, for example cliché became clik.
RL loop with a judge
The next stage used GRPO with a judge instead of an e-only reward. Each step sampled under the mask, then coherence and responds scored. Reward formula: coherence - 1 + responds - 1 divided by 8, producing 0 (both 1) to 1 (both 5). Group normalization makes an answer count only if it beats other answers to the same prompt. Consider step 5 as the illustration.
| Setting | Value |
|---|---|
| Prompts per step | 4 |
| Answers per prompt | 8 |
| Sampling temperature | 0.7 |
| Max answer length | 300 tokens |
| Thinking | off |
| LoRA rank | 16 |
| Learning rate | 2e-5 |
| KL penalty weight | 0.02 |
| Training steps | 300 |
- 4 prompts per step and 8 answers per prompt gave 32 answers; peak memory stayed within the 15 GB limit.
- GRPO needs varied scores; about two thirds of 8-answer groups had a usable answer.
- Temperature 0.7. At 1.0 the SFT adapter produced nonsense names like Ranak and palacco.
- The 300-token cap matches evaluation.
- Learning rate 2e-5 was lowered after overfitting.
- KL penalty 0.02 probably unnecessary.
- 300 steps covered 1,200 prompts from the 1,235-prompt pool; evaluations ran every 25 steps.
Starting points were checked on 50 new prompts with 16 tries each. The masked model got at least one usable answer for 78% of prompts; the SFT adapter got 64%. Both cleared the group-score condition, so I ran 300 steps from each.
Results and the RL curve
| Starting point | Before RL | After 300 steps | Gain middle 90% range |
|---|---|---|---|
| Masked | 26% | 43% | +17 (+8 to +26) |
| SFT adapter | 35% | 60.5% | +25.5 (+17.5 to +33.5) |
For stability, I re-drew the 50 prompts 1,000 times while keeping all 4 answers. The displayed band drops the lowest 5% and highest 5%, leaving the middle 90%.
| RL step | Masked | SFT adapter |
|---|---|---|
| 0 | 26% (19 to 33%) | 35% (27 to 43.5%) |
| 25 | 28% (20.5 to 36%) | 43% (34 to 52.5%) |
| 50 | 34% (25.5 to 43%) | 53% (44.5 to 62%) |
| 75 | 39.5% (31 to 48.5%) | 55.5% (47 to 64.5%) |
| 100 | 44% (35.5 to 53%) | 51% (42 to 60%) |
| 125 | 44% (34 to 53%) | 58.5% (50 to 68%) |
| 150 | 35% (26.5 to 44%) | 60% (52 to 68%) |
| 175 | 38% (29 to 48%) | 56.5% (47.5 to 65.5%) |
| 200 | 39% (30 to 47%) | 60.5% (52 to 70%) |
| 225 | 36.5% (28 to 45%) | 61.5% (53 to 70%) |
| 250 | 40% (32 to 48%) | 61.5% (53.5 to 69.5%) |
| 275 | 40% (31.5 to 48%) | 59% (50 to 68%) |
| 300 | 43% (34.5 to 51.5%) | 60.5% (51 to 69.5%) |
| 52 | 92 | 18 | 42 | 53 | 70 | 24 | 40 |
Final examples after RL
| Prompt | Before RL SFT adapter | After 300 RL steps |
|---|---|---|
| What is the movie Forrest Gump about | Repeats for a long time until the 300-token limit coherence 1 responds 1 | Low IQ man with war and family coherence 3 responds 4 |
| Introductory email to a lawyer about a ticket | Right to walk across a road coherence 3 responds 2 | Support to fight light violation coherence 5 responds 4 |
| Is there a Santa Claus and who is it | Man brings gifts with hat and job coherence 4 responds 3 | Fictional man brings gifts coherence 5 responds 5 |
| Who services Saint-Germain-les-Belles station with SNCF passage | Saint-Garmin-los-Ballons station rail hub run by SNCF coherence 4 responds 4 | SNCF coherence 5 responds 5 |
| Which sport is similar to baseball | Golf football rugby striking passing coherence 3 responds 2 | football coherence 5 responds 3 |
| Where does the word cliché come from | A clik is a loan from Dutch | Not shown in the source table |
Published by IT Insights, written by James Hartley, contact contact@itinsights.com.
For a General reader at ItInsightsWorld, the useful pattern is layered: the mask made the impossible constraint executable, SFT made e-free phrasing more natural, and judge-based RL raised the usable rate to 60.5%, while the shortest answers warn that reward shaping can trade detail for score.