Google opens the source for its robots.txt parser in Java and testing framework in C++

Posted on

Last year, Google open sourced the code for the robots.txt parser used in its production systems. After seeing the community build tools with it and add their own contributions to the open source library, including language ports of the original parser written in C++ to golang and rust, Google announced this week it has released […]