Package strit (STRing ITerator) assists in development of string processing pipelines by providing a simple
iteration model that allows for easy composition of processing stages.
Suppose we want to develop a function that reads a file line by line, removes leading and trailing
whitespace from each line, selects only non-empty lines that also do not start with the # symbol, and
stores those lines in a slice of strings. Using the Go standard library one possible implementation
of the function may look like this:
func ReadConfig(fileName string) ([]string, error) {
file, err := os.Open(fileName)
if err != nil {
return nil, err
}
defer file.Close()
var res []string
src := bufio.NewScanner(file)
for src.Scan() {
line := bytes.TrimSpace(src.Bytes())
if len(line) > 0 && line[0] != '#' {
res = append(res, string(line))
}
}
if err = src.Err(); err != nil {
return nil, err
}
return res, nil
}Using strit package the implementation can be simplified down to:
func ReadConfig(fileName string) ([]string, error) {
return strit.FromFile(fileName).
Map(bytes.TrimSpace).
Filter(strit.Not(strit.Empty).AndNot(strit.StartsWith("#"))).
Strings()- A number of iterator constructors for reading text from a variety of sources:
io.Reader:FromReaderFromReaderSFio.ReadCloser:FromReadCloserFromReadCloserSF[]byte:FromBytesFromBytesSFstring:FromStringFromStringSF[]string:FromStrings- Disk file:
FromFileFromFileSF - Directory listing:
FromDir - Recursive directory listing:
FromDirWalk - External command output:
FromCommandFromCommandSF
- Mapping and filtering primitives:
FilterGenMapMap - Sequence limiting functions:
SkipSkipWhileTakeTakeWhile - Search function:
FirstNonEmpty - Piping iterator output through an external command:
PipePipeSF - Iterator chaining (sequential combination):
Chain - Iterator merging (parallel combination):
Merge - Output collectors that invoke the given iterator and write the result to various destinations:
string:StringJoin[]string:Strings[]byte:BytesJoinBytesio.Writer:WriteToWriteSepTo- Disk file:
WriteToFileWriteSepToFile
- Predicates and predicate combinators for use with
Filter:EmptyStartsWithEndsWithNotAndAndNotOrOrNot - Basic parsing supported via
Parsefunction.
- Naïve
grep:
func main() {
_, err := strit.FromReader(os.Stdin).
Filter(regexp.MustCompile(os.Args[1]).Match).
WriteSepTo(os.Stdout, "\n")
if err != nil {
os.Stderr.WriteString(err.Error() + "\n")
os.Exit(1)
}
}- Recursively find all the filesystem entries matching the given regular expression:
func selectEntries(root string, re *regexp.Regexp) ([]string, error) {
return FromDirWalk(root, nil).Filter(re.Match).Strings()
}- Build a list of
.flacfiles in the given directory, annotating each name with its corresponding track number from FLAC metadata:
func namesWithTrackNumbers(dir string) ([]string, error) {
return strit.FromDir(dir, func(info os.FileInfo) bool { return info.Mode().IsRegular() }).
Filter(strit.EndsWith(".flac")).
GenMap(prependTrackNo).
Strings()
}
func prependTrackNo(file []byte) ([]byte, error) {
name := string(file)
no, err := strit.FromCommand(exec.Command("metaflac", "--list", "--block-type=VORBIS_COMMENT", name)).
FirstNonEmpty(func(s []byte) []byte {
if m := match(s); len(m) == 2 {
return m[1]
}
return nil
}).
String()
if err != nil {
return nil, err
}
if len(no) == 0 {
return []byte("???: " + filepath.Base(name)), nil
}
return []byte(no + ": " + filepath.Base(name)), nil
}
var match = regexp.MustCompile(`tracknumber=([[:digit:]]+)$`).FindSubmatchThe project is in a beta state. Tested on Linux Mint 19.1, with Go version 1.12. Should also work on other platforms supported by Go runtime, but currently this is not very well tested.