-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FR] support flag --benchmark_list_tests work with --benchmark-format=json #1642
Comments
In this one do, we have to modify the behavior of Line 540 in cbecc8f
In file if (FLAGS_benchmark_list_tests) {
if (FLAGS_benchmark_format == "json") {
// is this the palce to modify?
} else {
for (auto const& benchmark : benchmarks)
Out << benchmark.name().str() << "\n";
}
} else {
internal::RunBenchmarks(benchmarks, display_reporter, file_reporter);
} |
the test lists are a bit of a hack right now, and don't use the reporters at all. as such, i think you'll need to create a new |
This is what I added in the benchmark.h file - // Interface for custom benchmark result printers.
// By default, benchmark reports are printed to stdout. However an application
// can control the destination of the reports by calling
// RunSpecifiedBenchmarks and passing it a custom reporter object.
// The reporter object must implement the following interface.
class BENCHMARK_EXPORT BenchmarkReporter {
public:
//Remaining code
/**
* @brief Lists and describes the provided benchmarks.
*
* This virtual method is intended to be overridden by derived classes
* to provide specific implementations for listing benchmarks.
* It can be used for outputting, logging, or any other operation
* needed to handle or display the benchmarks' names and metadata.
*
* @param benchmarks A vector containing names and details of benchmarks
* that need to be listed or processed.
*/
virtual void BenchmarksList(const std::vector<BenchmarkName>& benchmarks) = 0;
private:
std::ostream* output_stream_;
std::ostream* error_stream_;
}; Now my question is about the implementation of this function, do I add that in the |
nit: it should be the implementation would be in all the *_reporter.cc files. the existing loop -> |
Tho I made the all changes which we are desiring - example - void JSONReporter::List(const std::vector<internal::BenchmarkInstance>& benchmarks) {
std::ostream& Out = GetOutputStream();
Out << "{\n \"benchmarks\": [\n";
for (size_t i = 0; i < benchmarks.size(); ++i) {
Out << " {\"name\": \"" << benchmarks[i].str() << "\"}";
if (i != benchmarks.size() - 1) Out << ",";
Out << "\n";
}
Out << " ]\n}";
} BENCHMARK_EXPORT
void ConsoleReporter::List(const std::vector<internal::BenchmarkInstance>& benchmarks) {
std::ostream& Out = GetOutputStream();
for (const auto& benchmark : benchmarks) {
Out << benchmark.str() << "\n";
}
} void CSVReporter::List(std::vector<internal::BenchmarkInstance>& benchmarks){
(void)benchmarks; // This is to prevent unused variable warning
std::ostream& err = GetErrorStream();
err << "List() method is not implemented for CSVReporter.\n";
} in Benchmark.cc if (FLAGS_benchmark_list_tests) {
if (FLAGS_benchmark_format == "json") {
JSONReporter().List(benchmarks);
} else {
for (auto const& benchmark : benchmarks)
Out << benchmark.name().str() << "\n";
}
} else {
internal::RunBenchmarks(benchmarks, display_reporter, file_reporter);
}
return benchmarks.size();
} But stuck to some errors. Maybe will try to do in the morning. |
I made this into because from the if (FLAGS_benchmark_list_tests) {
if (FLAGS_benchmark_format == "json") {
dynamic_cast<JSONReporter*>(display_reporter)->List(benchmarks);
} else {
//ConsoleReporter::List(benchmarks);
dynamic_cast<ConsoleReporter*>(display_reporter)->List(benchmarks);
}
} else {
internal::RunBenchmarks(benchmarks, display_reporter, file_reporter);
} I will look into this in the morning. Opening the PR so that you can better guide me over there. |
…tests-work-with-benchmark-format=json-google#1642'
Is your feature request related to a problem? Please describe.
For automation to collection the benchmark list easily, it is better to support use --benchmark_list_tests together with --benchmark-format=json.
Describe the solution you'd like
to output the benchmark list in json format.
Describe alternatives you've considered
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: