Faster utf8.Valid using multi-byte processing without SIMD.
-
Fast. Up to 50% faster than the standard library. See the benchmarks section.
-
Compatible. Its usage is the same as the standard library.
-
Well-tested. Fuzzing is actively done in addition to standard library's tests.
go get github.com/sugawarayuuta/charcoal
The left side is the standard library, the right side is this library (Higher is better).
│ ./before.txt │ ./after.txt │
│ B/s │ B/s vs base │
Valid/ascii-small-4 1.699Gi ± 2% 1.783Gi ± 1% +4.99% (p=0.000 n=10)
Valid/ascii-large-4 11.01Gi ± 1% 15.82Gi ± 1% +43.64% (p=0.000 n=10)
Valid/kanji-small-4 982.4Mi ± 1% 1099.9Mi ± 2% +11.95% (p=0.000 n=10)
Valid/kanji-large-4 890.4Mi ± 1% 1334.1Mi ± 1% +49.82% (p=0.000 n=10)
Valid/unicode.json-4 821.6Mi ± 7% 1226.3Mi ± 2% +49.26% (p=0.000 n=10)
geomean 1.658Gi 2.162Gi +30.44%
ascii-small: 10 ASCII characters.
ascii-large: 100K ASCII bytes.
kanji-small: 10 Japanese Kanji characters.
kanji-large: 100K Japanese Kanji bytes.
unicode.json: Contains many strings with various Unicode runes.
- The icon used is from ashleymcnamara/gophers.