Songgot (송곳): a Korean-first tiny agentic model for tool calling on the device

Hanish Keloth, Palette (Seoul and Bengaluru). Draft of 2026-09-10. Songgot rows in Table 2 are written from the run logs as each training run finishes; nothing on this page is estimated.

Abstract

Tiny agentic models (Needle 2, 45M parameters, 14 MB, July 2026) made on-device tool calling practical on phones and microcontrollers, but the released model is English only. Songgot is a Korean-first model in the same class: a decoder trained from scratch with a Korean-first 32k tokenizer, bilingual Korean and English pretraining on licence-clean text, and post-training on Korean tool-calling and structured-extraction data that no closed model touched. We evaluate on Kakao's FunctionChat-Bench SingleCall (500 Korean items) with a deterministic exact-match scorer that anyone can reproduce without an API key, against Needle 2, FunctionGemma-270M, Qwen3-0.6B and Qwen3.5-0.8B under identical prompts. Results for the comparators are measured and in Table 2; Songgot rows are filled from the run logs as each training run finishes (Songgot-nano on 2026-09-10). Weights, tokenizer, data generator, scorer and this paper are released under Apache 2.0.

1. Why a Korean-first tiny model

• The class exists and is growing: Needle 2 passed 52,000 downloads within weeks of release.

• On Hugging Face as of 2026-09-09 there is no Korean tool-calling model under 1B parameters; the one Korean function-calling finetune found is a 2B Gemma.

• Korean is expensive for English-first tokenizers. On the 100 FunctionChat SingleCall queries, Qwen3's 151k vocabulary spends 1.15 tokens per Hangul syllable, Gemma's 262k vocabulary 0.98, and Songgot's 32k Korean-first vocabulary 0.90 (Table 1). At a 256 to 1024 token budget shared with tool schemas, that is context.

2. Model

Llama-style decoder, hidden 512, GQA with 8 query and 2 key-value heads, SwiGLU intermediate 1408, RoPE, tied embeddings, 32k vocabulary, trained from scratch. Two sizes: Songgot-nano, 8 layers, 39M parameters, pretrained on an Apple M5 Max with MLX (320M tokens); Songgot, 12 layers, about 50M parameters, 8xH100 run on 6B tokens queued behind GPU budget. Exports to safetensors, GGUF (f16, Q8_0, Q4_K_M) and runs under llama.cpp.

3. Tokenizer

SentencePiece BPE, 32,000 pieces, byte fallback, digits split, trained on a 300 MB sample: Korean Wikipedia 50 percent, fineweb-edu 40 percent, tool-schema JSON 10 percent. Special tokens <|system|> <|user|> <|call|> <|end|> <|pad|>. Post-training, evaluation and the app share one tokenizer: llama.cpp's, driven through its Python bindings with a vocabulary-only GGUF (the five specials typed as user-defined tokens). SentencePiece folds newlines into spaces and inserts a space piece before each special; llama.cpp does neither, and it is what every GGUF user runs, so the model is trained and scored on those ids.

Table 1. Tokens per Hangul syllable on the 100 FunctionChat SingleCall queries (2,071 syllables).

4. Data

Pretraining (all disclosed, all licence-clean):

• fineweb-edu sample-10BT (ODC-By), first 1.63B tokens under our tokenizer.

• Korean Wikipedia, dump 20231101.ko (CC BY-SA 3.0), 0.60B tokens, upsampled to roughly half of the training mix.

• No AI-Hub data (its terms forbid leaving Korea and our GPUs do not sit there), no crawled Korean web text of unclear licence.

Post-training (101,464 examples, set v3): 64,000 Korean tool calls generated from hand-written frames for 57 Korean tools (messaging, calendar, transit, home appliances, commerce, public services, health, work, information), with register variants (반말, 존댓말, 격식, 사투리, typos) and Korean date and time words resolved against a fixed calendar; 8,083 Korean calls generated by our own Palette-K-Midm (open weights) as teacher; 27,948 English calls from glaive-function-calling-v2 (Apache 2.0) over thousands of distinct tools; 6 percent "no tool applies" negatives. Each prompt carries 1, 3 to 6 or 8 tools; half of the multi-tool Korean prompts use close distractors (tools from the target's own category), mirroring the benchmark's conditions.

Tool-name style augmentation, the change that mattered most: in 52,165 rows every tool is renamed in a random style (camelCase, PascalCase, snake_case, digit or word suffixes, verb synonyms; 47,192 distinct names in the set) and the target call is renamed to match. Our first 12-layer model, trained without it, scored 0.2 percent: it copied any invented snake_case name exactly and could not copy a single camelCase one, because every name it had seen was snake_case (the benchmark mixes both). A tiny model learns the name style it is shown; showing it every style teaches it to copy the name from the prompt. Two related fixes: glaive's arguments are single-quoted and 97 percent of its calls had failed to parse in earlier sets (the English part was 3,519 rows and mostly negatives), and "no tool" was reduced from 28 to 6 percent because the benchmark never asks for it. No closed model produced a label. A disjointness gate aborts the build if any training tool name or query appears in FunctionChat-Bench.

Two later additions (sets v4 and v5, same day): 7,745 Korean rewrites of glaive requests produced by our own Palette-K-Midm with the schemas and gold calls left untouched (Korean queries over thousands of tools instead of 57), no-parameter tools upweighted, and every no-parameter schema rendered in one of the three common JSON shapes at random. The last one came from reading failures: 45 of the benchmark's 57 no-parameter tools use {"type":"object","properties":{},"required":[]}, our set had almost only bare {}, and the model had learned the shape instead of the rule. Each set was scored before the next was built; Table 2 carries the published one and the text names the others.

v6 (2026-09-10 evening) attacks the real limit of v3-v5: those sets contain only 645 distinct real tools (the rest of their 47k names are restylings), while the benchmark's 25 tools are all new to the model. Our own teacher (Palette-K-Midm, no closed model) invents tool schemas for 60 Korean service domains in six naming styles, writes three Korean requests in different registers for each tool together with the exact call, and then, in a second pass with three distractor tools present, re-derives the call from the request alone; a pair is kept only when tool name, argument keys and values agree (99 percent agree on the name, 58 percent on everything). Three runs gave 19,293 invented tools and 36,216 verified rows; with close distractors drawn from the same domain, 30 percent of rows restyled and 5 percent turned into negatives, v6 has 152,494 rows over 113,197 distinct tool names. On the same 6B-token base, v6 lifts call accuracy from 11.4 to 23.8 percent and name accuracy from 53.6 to 63.8: the single largest step in this paper, and it came from data, not parameters or RL. A fourth synthesis run of 30 passes (v7: 100,363 verified rows over 52,483 invented tools, 219,822 rows in all, 223,319 distinct names) lifts the same base to 27.6 percent call and 69.4 name accuracy, with the close-distractor conditions moving most (4_close 13 to 22). A fifth run of 60 passes added a sibling stage: for 40 percent of invented tools the teacher writes four confusable tools in the same service (similar names, different functions), and verification shows siblings first among the distractors. v8 (336,498 synthesised rows over 183,713 tools, 467,843 rows in all, 642,439 distinct names) reaches 32.0 percent call and 71.4 name accuracy on the 6B-token base (31.0 and 72.2 on the pre3 base, a tie), with 8_close moving from 13 to 20. Three runs of the same recipe took the model from 11.4 to 32.0 percent; the curve is still rising at about four points per tripling of the synthesised set. A third epoch over v8 gives 33.0 percent (name 73.6), inside the error bar of the two-epoch run but not below it, and is the published 12-layer model. Songgot-M post-trained on the same v8 set scores 33.2 (name 73.8): with 2.5 times the parameters and the same data the two models tie, for the third time in this paper. A sixth synthesis round of 120 passes (v9: 755,011 synthesised rows over 404,424 invented tools, 907,268 rows in all) scores 33.0 percent (name 72.6): a tie with v8. The data curve, which gave about four points per tripling from v5 to v8, is flat from v8 to v9. On this 6B-token base the synthesised data has done what it can; the remaining gap to the pretrained comparators is pretraining, and the next runs change the pretraining corpus, not the post-training set. The synthesised pool is released as hf.co/datasets/palette-lab/songgot-tools-ko.

Post-training recipe: full-parameter SFT, one epoch then a second at a lower rate, followed by a similarity-reward RL stage (group-relative policy optimisation against the gold call, reward = format term plus argument similarity as in STAR, Ni et al. 2026, with our own labels as the only signal). Each stage is scored on the benchmark and a stage that lowers the score is not published; Table 2 reports the published one. For the 12-layer model the RL stage tied the SFT score (11.4 percent before and after), and the published weights carry it.

What the RL stage measured. Three runs on the 12-layer model, 16 prompts x 8 samples per step, KL to the frozen SFT model. Run 1 drew prompts from the post-training set: the group mean reward was 0.97-0.99 over the first 100 steps, the eight samples of a group almost always agreed, and the benchmark did not move. Run 2 drew the same queries with every tool renamed to a name that never appears in training: reward 0.84-0.99 over 140 steps, because the model reproduces the gold call for a query it was trained on even when the tool is called something else. Run 3 used prompts the model had never seen (1,575 Korean rewrites of glaive requests held out of post-training, their renamed copies, and 3,000 held-out English rows): reward 0.70-0.85 with exact matches at 0.58-0.69, and real disagreement inside each group, the first run with a gradient worth following. After 200 of those steps the benchmark was still 11.4 percent, identical in every condition. A model this small memorises its post-training queries, so on-policy RL can only teach what SFT has not already fixed, and 200 steps of it on 3,200 novel prompts did not reach the benchmark's tools. The RL stage is kept in the recipe as a measured negative, not as a source of the published number.

5. Evaluation

Korean comparators, measured 2026-09-11 with the same prompts, scorer and each model's own chat template: Kakao's Kanana-2-1.3B-Instruct scores 73.2 percent, LG's EXAONE-4.0-1.2B 63.0. These are the Korean state of the art at on-device size and the numbers to beat; Kakao also authored FunctionChat-Bench, so Kanana's figure may reflect in-distribution training. DNA3.0-0.8B receives a prompt byte-identical to its base model Qwen3.5-0.8B (45.2 percent) and scores 4.8: it answers 421 of 500 items in prose, so its post-training removed most of the base model's tool calling. HyperCLOVA X SEED 0.5B has no tool interface in its chat template (the tools never reach the prompt), so it is listed without a score rather than with a zero.

Parser leniency, adopted 2026-09-14 and applied to every model: a tool call emitted as a bare <function=...>...</function> block without its <tool_call> wrapper is parsed (first block wins). It changed Kanana-2 from 70.4 to 73.2 and Songgot-X from 60.8 to 61.8 and no other row; all numbers in this paper are under the lenient parser.

Error bars first. With 500 items the standard error of a call-accuracy figure near 25 percent is 1.9 points, so two models within about 4 points of each other are not separated by this benchmark; we treat such differences as ties and say so where they occur.

FunctionChat-Bench SingleCall (Kakao, 2024, Apache 2.0): 25 functions, 4 Korean queries each, 5 tool conditions (exact, 4 random, 4 close, 8 random, 8 close) = 500 items. The official protocol uses GPT-4 as judge; we score with exact match on function name and arguments (numbers compared as numbers, acceptable alternatives honoured) so the number is reproducible offline. Every model receives the same tools and query, rendered in its own documented format.

Table 2. Call accuracy (exact match) on SingleCall, by tool condition, percent. Songgot rows are written by the build from the run logs; a row reads "training" until its run has finished.

6. Honest reading

• Songgot-nano, 320M pretraining tokens and one post-training epoch on the v3 set, scores 0.0 on both call and name accuracy. It answers with tool names from its own post-training catalogue (create_event, get_subway_arrival, lookup_exchange_rate) instead of the names in the prompt, and "none" for 105 of the 500 items. The 12-layer model pretrained on 6B tokens copies the right name 54 percent of the time from the same data. At 320M tokens the model has not learned to copy from context at all; the Needle 2 recipe used 200B. The nano weights stay published as the measured floor of the recipe, and the 12-layer model is the one in the app.
• Songgot-M, 16 layers and hidden 768 (126M parameters), pretrained on the same 6B tokens and post-trained on the same v5 set, scores 12.0 percent against the 12-layer model's 11.4: 32 against 26 percent with a single tool, and the same or worse with four and eight tools (13/8/7/0 against 12/10/7/2). Name accuracy is 53.0 against 53.6. Two and a half times the parameters buy three more correct calls out of 500, all of them in the easiest condition. At this token count the bottleneck is what the model has read, not how many parameters it has, and the next experiment is more pretraining tokens on the 12-layer size, not a larger model.
• More tokens of the wrong kind hurt. A second corpus (24.4B tokens: 9.0B of FineWeb-2 Korean web text, the 0.6B of Korean Wikipedia, 14.8B of fineweb-edu; same tokenizer) trained the 12-layer model for 24B tokens, four times the first run and fifteen times its Korean. Post-trained on v5 it scores 8.6 percent against 11.4 for the 6B-token model on the same data, and on v6 17.4 against 23.8. Name accuracy rose (55.8 and 66.8 against 53.6 and 63.8) while whole-call accuracy fell: the bigger corpus picks the tool a little better and fills its arguments worse, with invented arguments on tools that take none. For a tool-calling model of this size the encyclopedic and educational mix of the first corpus transfers better than general Korean web text, and the next pretraining run keeps that mix, trains longer on it, and adds a small share of tool-calling rows in the pretraining stream.
• What our data is worth on a pretrained base, as a reference we do not ship: Qwen3-0.6B (Apache 2.0) post-trained on the v5 set with its own chat template, two epochs, scores 55.4 percent against 43.2 zero-shot (name accuracy 86.6). The same v5 set gives the 50M from-scratch model 11.4, and v7 gives it 27.6. The remaining gap is what 600M parameters and trillions of pretraining tokens buy; the from-scratch line is the one we ship, and the data lever is the one that keeps moving it.
• Instruction rows in the pretraining stream help a little. A third pretraining run kept the first corpus, took three passes over it (12B tokens) and mixed in 5 percent of v6 rows rendered exactly as post-training renders them. Post-trained on v6 it scores 24.6 percent against 23.8 for the 6B-token base; post-trained on v7 it scores 25.0 against 27.6. Both differences sit inside the benchmark's error bar (section 5), so the two bases are a tie on this test, and v8 is trained on both.
• Six times the parameters on the wrong corpus lose. Songgot-L (24 layers, hidden 1024, 303M parameters) was pretrained from scratch for 24B tokens on the second corpus, the general Korean web mix that had already cost the 12-layer model six points, and post-trained on v8: 28.2 percent call, 72.4 name (published as palette-lab/songgot-l with that number). The 50M model on the first corpus scores 33.0. Size and corpus are confounded in that comparison, so the same 303M model is being pretrained on the first corpus with the instruction bucket (29.4, next bullet); at this scale the corpus has mattered more than the parameter count every time we have measured it.
• Our own documents do not help pretraining either. A third corpus mixed 30 percent document text (Palette's private Korean documents, 0.73B tokens, and the ko-vdr page markdown, 0.19B) into the first corpus with the instruction bucket, 12B tokens on the 12-layer model: 24.2 percent call after v8, against 31.0 for the same recipe without the documents and 33.0 for the plain 6B-token run. Every pretraining change we have tried on the 12-layer model, more passes, general web text, document text, six times the parameters, instruction rows in the stream, has tied or lost against the original 6B tokens of Korean Wikipedia and fineweb-edu. For a tool-calling model of this size, pretraining text is not the lever between 33 and 63; the comparators that score there have read three orders of magnitude more text than we have.
• A copy-training bucket does not fix argument copying. Since 203 of the 500 misses were the right tool with a garbled copy of a query span as the value, we generated 300,000 pretraining exercises that copy a Korean span verbatim from context (particle stripped) and mixed them with the tool-calling rows at 15 percent of the 6B-token stream. Result after v8: 29.6 percent; wrong-tool errors fell from 127 to 116, but right-tool-wrong-value errors rose from 203 to 229. The skill transferred to the pretraining task and not to the benchmark. With this, the 12-layer recipe is at its measured ceiling of 33 percent, and we stop spending compute on it until a large clean Korean text corpus is in hand; the paper will say when that changes.
• Songgot-V after a multiple-choice stage (113,844 questions rewritten as four-way choices, 2,500 steps) scores K-DTCBench 0.258 and K-MMBench 0.294: still chance. The loss with a shuffled image equals the loss with the right one during that stage, so the model learned the answer format from the text and not from the page. Reading a page (stage 1) and reasoning over it in a benchmark format are different skills, and a 50M language model has not shown the second one. The vision line waits for a larger language model.
• The same 303M model on the first corpus with the instruction bucket, 12B tokens, post-trained on v8: 29.4 percent (name 75.0), against 31.0 for the 12-layer model on the identical corpus recipe and 33.0 for the published 12-layer model. Two controlled pairs now agree: at this token budget more parameters do not raise call accuracy, and the corpus decides. Published as palette-lab/songgot-l with that number.

• Line B, for scale: Qwen3.5-0.8B post-trained on 200,000 v8 rows for one epoch with its own chat template scored 61.8 percent (name 94.8), 16.6 points above its zero-shot 45.2 and 1.2 below EXAONE-4.0-1.2B; a 2B base on the same rows scores 61.2 and the full 468k rows 58.6, so the data, not the base, sets this line's ceiling. The targeted v10 set (below) took the same base to 67.4 (name 93.6), a uniform weight average of the v10, v11 and v12 checkpoints to 78.2 (name 97.0), and the average of that soup with a short continued fine-tune on v13 rows to 82.2 (name 96.4), 9.0 points above Kanana-2-1.3B on call and within a standard error on name. That average is published as Songgot-X with the base named on the card; it is a product, not a from-scratch claim, and it shows what the data is worth to a model that has read trillions of tokens.

• The template-generated Korean data is narrow by construction; a teacher-generated set from our own Palette-K-Midm was planned and is queued behind GPU availability.

• The preview checkpoint used to open the demo on 2026-09-10 is an early one (65M tokens, 24,000 post-training examples) and is labelled as such in the table.

Copy-consistent subset (line B, 2026-09-14). We trained the same Qwen3.5-0.8B recipe on the 306,868 v8 rows in which every argument value appears verbatim in the query (200,000 sampled, one epoch, booleans typed as JSON). Exact match fell to 58.6 call / 88.4 name, from 61.8 / 94.8 on the plain v8 sample. Removing the rows where the gold value is a normalisation of the query (07:00 for 아침 7시, ISO dates, canonical city names) removes exactly the behaviour the benchmark rewards, and the smaller name set lost tool coverage. The published Songgot-X stayed on the v8 weights until the v10 round below replaced them.

Targeted round v10 (line B, 2026-09-15). Reading the v8 Songgot-X misses against Kanana-2 put the gap in four classes: argument values not copied verbatim from the query (56 items), a wrong tool chosen among close siblings (15), Korean number words (10) and two-digit years (5). We wrote 31,307 synthetic rows for the copy, from/to role, number-word and year cases with our own teacher (Palette-K-Midm) and kept only rows that passed deterministic checks (the value is in the query; the number word converts to the number; the two-digit year expands to the stated one), added the 10,861 human requests of the MASSIVE 1.1 ko-KR train split (CC BY 4.0) mapped onto the massive-agents function declarations, and trained the same 0.8B recipe on 263,242 rows: 175,000 sampled from v8, the synthetic and MASSIVE rows at weight 2, KoSGD, single-tool rows given 1, 4 or 8 distractors (name-related or random), 30 percent restyled, 5 percent turned into negatives, benchmark names and queries excluded by assertion. Exact match rose to 67.4 call (from 61.8) and fell to 93.6 name (from 94.8); the standard error is 2.1 points, so the gain is outside noise and the 5.8-point gap to Kanana-2 remains. Item by item: 70 fixed, 42 lost, 267 unchanged. Wrong-argument misses fell from 165 to 131 and wrong-tool misses from 25 to 19; the two functions that gained most, generate_random_password (+19 items) and start_playlist (+15), are the ones whose arguments are copied or counted. The losses say what the data still lacks. count_words lost 19 items: the copy rule taught copying, and the model now copies the instruction prefix ("이 문장 몇 단어인지 세줘: ...") into the text argument where the v8 model took only the payload; the synthetic set has no instruction-plus-payload rows. Thirteen items are declined ("no matching tool") although the tool is offered, against one before, which is where the name score went. One date in 1989 was written as 2026, because no row in the set carries a 19xx year. Each is a gap rather than a wrong row, and each has a deterministic fix; that is the next round. The v10 weights replaced the v8 weights at palette-lab/songgot-x-0.8b on 2026-09-15 (safetensors and GGUF, headers repaired by the exporter); the score JSON is eval/score_q35_08b_v10.json.

Held-out human utterances (MASSIVE ko-KR test, 2026-09-15). FunctionChat is 500 items on 25 functions written by one team; a second test with other authors and another convention guards against tuning to one benchmark. We took 500 utterances from the MASSIVE 1.1 ko-KR test split (human-written voice-assistant requests, CC BY 4.0; the dev and test splits never entered any training set, and test utterances that also occur in train were removed), gave each its gold massive-agents function plus 3 random ones of the 55, and scored exact match on the function name and on the full call after whitespace normalisation (MASSIVE's values are spoken spans). Subset bench/massive_ko_test_500.jsonl, scorer eval/score_massive.py, predictions from harness/eval_massive.py.

The name column is the fair comparison: the v10 member and Kanana-2 choose the same function at the same rate (81.4 against 83.4, inside the 1.7-point standard error), post-training lifts the base by 18 points, and the published three-way average reaches 94.0, 11 points above Kanana-2 on human requests it never saw, which says the average generalises rather than fits the benchmark; its call score (70.4) sits with the other averages inside noise. The call column is not a fair comparison and we do not present it as one. Songgot-X trained on the MASSIVE train split, so it knows this convention (copy the spoken span verbatim, "오전 다섯 시"); Kanana-2 never saw it and fills arguments with placeholders ("your_list_name", "user's location", "today") or declines. That is the mirror image of FunctionChat, where Kakao's authorship gives Kanana the convention. Read together, the two tests say that convention, not Korean understanding, decides most of the argument gap between these two models, and that a 0.8B model reaches parity on the function-choice part. MASSIVE's ko-KR slot annotation is noisy (spans such as "요일이니" for a date), which caps every row alike.

Round v11 (2026-09-15, not published). Two data gaps from the v10 reading, filled with 3,774 teacher rows checked by deterministic rules (harness/teacher_synth_v11.py): instruction-plus-payload requests whose text argument must carry the payload only, and requests with an explicit 1950-2015 year. MASSIVE at weight 1 and negatives at 2 percent. Result 69.2 call / 94.4 name, against 67.4 / 93.6 for v10, inside the 2.1-point standard error. The targets moved exactly as intended: count_words 0 of 20 to 20 of 20, refusals of an offered tool 13 to 1, the 1989 date kept. The gain was eaten elsewhere: wrong-tool choices rose from 19 to 27, almost all close siblings (start_playlist taken as create_playlist, informWeather as getWeatherForecastForNext7Days), one optional argument invented (shuffle: false) and one location narrowed (제주도 서귀포시 to 서귀포시). 54 items fixed, 45 lost. Those swings sit on functions the two runs never targeted, so we read them as run-to-run variance from the different 175k sample of the v8 set rather than as an effect of the new rows, and we test that reading two ways: a uniform weight average of the v10 and v11 checkpoints (harness/soup_base.py), and the v12 round below. The v11 weights are not published on their own.

Weight soup of v10 and v11 (2026-09-15). The two checkpoints share the base, the recipe and 175,000 of their rows and differ in the sample and the targeted rows, so their weights sit in one basin and can be averaged (uniform, in fp32, harness/soup_base.py; no training). The average scores 74.0 call / 96.0 name: 43 items that v10 missed are fixed and 10 lost, 39 fixed and 15 lost against v11. Every per-function swing between the members closes toward the better member (start_playlist 15 and 9 to 19 of 20, informWeather 20 and 16 to 19, get_exchange_rate 20 and 16 to 20, count_words 0 and 20 to 20), wrong-tool choices fall back to 20 and no item is unparsed. The union of items either member gets right is 391; the soup gets 370, so it recovers most of what the two runs knew separately. Against Kanana-2-1.3B (73.2 / 97.2) the call score is 0.8 higher and the name score 1.2 lower, both inside the 2.0-point standard error: a tie, stated as such. The soup replaced the v10 weights at palette-lab/songgot-x-0.8b on the evening of 2026-09-15; score JSON eval/score_soup_q35_08b_v10v11.json.

Round v12, follow the parameter description (2026-09-16). The misses left after v11 were conventions the schema itself states: AddAlarm.time says "사용자의 표현 그대로 추출" and the model normalised spacing; CreateEvent.start_datetime gives no format and the model wrote ISO, because our v10 year rows had taught ISO regardless of the description; memo is optional and the model filled it; QueryCalendar.query says "일정의 이름" and the model kept "이번달"; send_message wants the message as the recipient reads it and the model copied reported speech. Five teacher modes with deterministic rules (harness/teacher_synth_v12.py, 3,309 rows kept): a format hint injected into the stored schema and obeyed; a datetime parameter without a hint copied verbatim (every token of the value in the request, so range ends may repeat the date); optional parameters omitted; a value narrowed to the named thing; a message rewritten into direct speech (verified on tool choice only, since two free rewrites never agree word for word; the first pass kept 17 of 6,003 under full agreement, the second 1,400). The v10 year rows were re-labelled with a "(YYYY-MM-DD)" hint so ISO is tied to a stated format. Trained alone on the same recipe (266,547 rows, seed 12): 75.2 call / 93.2 name, the first single checkpoint above Kanana-2 on call. The targeted classes moved (send_message 0 to 15 of 20, QueryCalendar 11 to 20, addMemo 12 to 15) but CreateEvent stayed at 0 because the model still fills the optional memo with the time span, AddAlarm only 5 to 7 ("3시간 뒤" still loses 뒤), and wrong-tool choices among close siblings rose to 34 (convert_currency taken as convert_eur_to_jpy, informDday as search_contact), which is what the RL stage is for.

Three-way soup, the published weights (2026-09-16). The uniform average of v10, v11 and v12 scores 78.2 call / 97.0 name and is even across conditions (80 / 81 / 77 / 77 / 76). It gets 391 items right against 366 for Kanana-2: 61 that Kanana misses, 36 the other way, 330 both. Wrong-tool choices fall to 15 (Kanana 10), argument misses to 94 (Kanana 120). Per function the two models have different strengths: Songgot-X wins generate_random_password (15 of 20 against 0), start_playlist (20 against 0), convert_currency (12 against 3) and getCurrentTimeForLocation (20 against 15); Kanana wins AddAlarm (15 against 5, the verbatim time expression), QueryCalendar (20 against 16), informDday (18 against 14) and update_contact (13 against 10). The average of v11 and v12 alone scores 78.0 / 96.2 and the earlier v10 and v11 average 74.0 / 96.0; each added member closes more of the union of what the members know. Standard error at 78 percent on 500 items is 1.9 points, so 5.0 points above Kanana-2 on call is about 2.6 standard errors and outside the 4-point band we set for ties in section 5; the name scores (97.0 against 97.2) are a tie. Score JSON eval/score_soup_q35_08b_v10v11v12.json.

GRPO on line B, two rounds, no change (2026-09-16). We ported the similarity-reward GRPO stage of section 4 to the base model's chat template (harness/rl_base.py: reward -1 for no call, 0 for the wrong tool, else mean per-key similarity with extra and missing keys counted against; KL 0.02 to the frozen soup; lr 1e-6). Round 1 sampled 16 prompts x 8 completions per step from 24,000 v8 rows that none of the three members had seen; the soup already answers 85.8 percent of them, and at temperature 0.8 the eight samples of a prompt were identical, so most steps had no group with reward variance and the benchmark did not move (78.4 / 97.0 after 300 steps). Round 2 kept only the 3,501 of 24,000 prompts the soup gets wrong under greedy decoding (2,119 wrong tool, mostly close-sibling conditions; 1,381 wrong argument) and sampled 8 x 16 at temperature 1.0. Now every step had signal and the mean reward on those prompts rose from 0.38 to about 0.8 across 300 steps, but the benchmark still did not move: 78.2 / 96.8, one item gained and one lost against the soup. What the policy learnt on its own hard prompts did not transfer to Kakao's items, and we did not raise the learning rate or lower the KL weight to force it, since the from-scratch runs (section 4) had already shown that this reward mainly re-teaches what supervised data teaches more cheaply. The two RL checkpoints are not published; every gain in this section came from targeted data and weight averaging.

Round v13, continued fine-tuning on the soup (2026-09-16). The misses left in the three-way soup where Kanana-2 still led were four: the alarm time copied with normalised spacing ("10분 후" for "10분후", "3시간" for "3시간 뒤"), the optional memo filled with the time span, old/new roles in update calls, and a query-type request answered with a create-type tool. Rows for all four came from the teacher with deterministic rules (harness/teacher_synth_v13.py: 1,622 verbatim time rows where the value must be an exact substring of the request and must keep a following 뒤/후/전/반; 937 old/new rows; 254 intent rows; memo omission built deterministically by adding an unfilled optional memo/note parameter to the v12 datetime schemas, 1,971 rows). Instead of a fourth 10-hour run from the base, we continued the published soup itself for one epoch on the new rows (twice, with distractors) plus a 25,000-row replay sample of the v12 set at a halved learning rate (5e-6, 958 steps, about one hour), then averaged that continuation with the soup. The continuation alone scores 80.8 / 95.4: AddAlarm goes from 5 to 20 of 20 and argument misses from 94 to 73, but wrong-tool choices rise from 15 to 23 (informDday 14 to 8), the pattern of every targeted round so far. The average with the soup keeps the argument gains and most of the tool choice: 82.2 call / 96.4 name, 411 items right (27 gained and 7 lost against the soup), 18 wrong tools and 71 argument misses. Against Kanana-2 it is 74 items to 29 with 337 shared; Kanana-2 still leads on informDday (18 to 12), update_contact (13 to 10), addMemo (20 to 17) and add_task (17 to 15). A second continuation on the memo rows scored 81.4 / 95.4 alone, 82.0 / 96.4 averaged three ways, and 82.8 / 96.8 averaged with the soup alone, three items above the published weights and inside the standard error; we keep the published weights rather than churn the release for noise, and note the candidate here. On the held-out MASSIVE subset the published average scores 72.0 call / 94.0 name (three-way soup 70.4 / 94.0; predictions for this row came from the batched left-padded decoder, which agrees with the single-prompt path up to bf16 batching noise). Score JSON eval/score_soup_s3_c13.json.

6b. Songgot-V, first numbers

Songgot-V pairs the from-scratch language model with an open pretrained vision tower (SigLIP2 base, 512px, Apache 2.0, trained by Google and stated as such) through a projector trained from random initialisation; the page is cut into six 512px tiles plus a thumbnail, 448 image tokens. Stage 1 aligned the projector on 245,791 Korean document pages (ko-vdr, CC BY 4.0); the loss with the right page was 3.67 against 5.54 with a shuffled page, so the model reads the image. Stage 2 trained projector and language model on 113,844 teacher-written Korean questions and answers about those pages. Scored on 2026-09-12 with next-token likelihood over the option letters, the same protocol as our baselines: K-DTCBench 0.275 (document 0.375, table 0.175, chart 0.275) and K-MMBench 0.269, against a 0.25 chance floor and 0.610 to 0.783 for the 450M to 1.7B comparators. The model had never seen a lettered multiple-choice question; a stage teaching that format is running, and these numbers stay in the paper either way.

7. Prior art and what we claim

A sweep of the Hugging Face Hub, Korean press and Korean search on 2026-09-11 (docs/PRIOR_ART_2026-09-11.md in the repository) rules out most of the sentences one would like to write, and we record them here so that nobody else has to. Korean tool calling is not new: Kakao published FunctionChat-Bench in 2024 and ships tool calling in Kanana-2-1.3B (2026-07), and LG's EXAONE-4.0-1.2B (2025-07) does agentic tool use. Tiny from-scratch Korean models are not new: KAWK-1.5-50M (51.5M, 2026-08-03), haru (6.8M, 2026-08-11) and SKT's KoGPT2 (125M, 2021) all predate this work, and KoGum-0.5B (2026-03) and KoHRM-Text-1.4B (2026-05) are from-scratch Korean models with tool-call support. Songgot is not the best Korean tool-calling model at any size: on our own scorer Qwen3.5-0.8B and Qwen3-0.6B, neither Korean-made, score 45.2 and 43.2 against our 33.0 and 33.2.

What we found nothing to contradict, and therefore claim, with its qualifiers: Songgot is the smallest openly licensed (Apache 2.0) Korean model trained from scratch for tool calling with a published FunctionChat-Bench SingleCall number, at 50M and 126M parameters, and it runs offline in a browser. Our numbers are exact-match under our own scorer, not the benchmark's GPT-4 judge, and we say so wherever they appear. Korean leaderboards (Dnotitia, Horangi, NIA K-AI) bracket models at 27B or 30B and below and have no on-device or sub-1B category, so "ranked" claims are not available to a model of this size in Korea today. For the vision model in preparation we found no Korean vision-language model under 1B parameters; we will say "we found none", not "none exists". A Naver-native sweep on 2026-09-12 (16 queries through Naver's own index, docs/NAVER_SWEEP_2026-09-12.md) found the same picture and one closed product to note: 뉴플로이 (Newploy) shipped an on-device Korean document OCR service on 2026-08-19 without weights or benchmark numbers. Kakao's Orchestration Benchmark (ICLR 2026) is a second Korean agentic benchmark on which no model under 2B parameters has been scored.

What line B claims (2026-09-16, evening). On FunctionChat-Bench SingleCall under our exact-match scorer, Songgot-X (0.8B, Apache 2.0) scores 82.2 call / 96.4 name: 9.0 points above Kanana-2-1.3B-Instruct (73.2 / 97.2) on call, about 4.7 standard errors and well outside the 4-point band this paper treats as a tie, and within one standard error on name. It is the highest score of any model we measured on this benchmark, and the highest openly licensed one by a wider margin; Kanana-2 cannot be redistributed on device under its licence and EXAONE-4.0 is research-only. Qualifiers that travel with the claim: one benchmark, 500 items, our exact-match scorer rather than the official GPT-4 judge, Kakao authored the benchmark, and the date. On the held-out human test (section 6) the function-choice column, the fair one, favours Songgot-X as well.

8. Release

Weights and tokenizer: hf.co/palette-lab/songgot-12l (and songgot, songgot-m). Synthetic data: hf.co/datasets/palette-lab/songgot-tools-ko. Code, data generators, scorer, paper: github.com/hanishkeloth/songgot. Licence Apache 2.0. Korean Wikipedia attribution and CC BY-SA notice in the model card.

9. On the device

Songgot Pocket (hanishkeloth.github.io/songgot/app) is the 12-layer model running inside the browser: llama.cpp compiled to WebAssembly (wllama 3.6.1), the Q8_0 GGUF (54 MB) fetched once and kept in the browser cache, no server in the loop. It installs as a web app on iOS, Android and desktop and keeps working with the network off. The app shows the model at most six candidate tools per request (chosen by character bigram overlap with a 58-tool Korean catalogue), renders the returned call as a card, and performs the calls it can do locally (alarms, timers, notes, calendar files) while handing the rest to the right site when online. Measured on 2026-09-10 in Chrome on an Apple M5 Max, single thread: 1.5 s per request from prompt to parsed call with the 12-layer weights (755 prompt tokens, 31 generated, 21 tokens per second), 1.7 s with the 39M weights.

Songgot-X runs in the same app. Measured on 2026-09-15 with the v10 weights in Chrome on the same machine, single thread, Q4_K_M GGUF (505 MB): 2.0 to 2.2 s per request from prompt to parsed call (742 to 764 prompt tokens, 28 to 41 generated, 13 to 21 tokens per second; the v8 weights measured 3.5 s at 12.2 tokens per second on 2026-09-14). Two defects had to be removed first, and neither is specific to our weights. The llama.cpp converter reads mtp_num_hidden_layers = 1 from the Qwen3.5 config and declares 25 blocks plus a next-token-prediction layer, although the fine-tuned checkpoint holds no such tensors (transformers drops them), so the file loaded nowhere ("tensor blk.24.attn_norm.weight not found"); rewriting the header to the 24 blocks that exist fixes it, and the exporter now does this itself. And the 248,320-entry vocabulary makes the logits buffer vocab x n_batch x 4 bytes, 508 MB at the runtime's default batch of 512, which the single-thread WebAssembly heap cannot allocate ("Invalid typed array length"); a batch of 64 keeps it at 64 MB. The official Qwen3.5-0.8B Q4_0 file shows the same behaviour in the same runtime (19.8 tokens per second at batch 32), which is how the second defect was separated from the first.