Repository navigation
Refactor the CBBO and DTLZ benchmarks #32
Description
Activity
Is uv or conda used for development of this benchmark package?
- changed the title
[-]Refactor the DLTZ benchmarks[/-][+]Refactor the DTLZ benchmarks[/+]on Mar 17, 2025 I used
uv syncfrom the root of this repository.- changed the title
[-]Refactor the DTLZ benchmarks[/-][+]Refactor the CBBO and DTLZ benchmarks[/+]on Mar 24, 2025 A benchmark will now be defined as a subclass of
HPOBenchmark(see definition)An example usage will be:
from deephyper_benchmark.benchmarks.cbbo import AckleyBenchmark as bench search = CBO(bench.problem, bench.run_function) results = search.search(max_evals=100)
- cbbo
- dtlz @Deathn0t
The logic to add noise to the objective or to add a sleep time to the objective can be factoried as
deephyper_benchmark.utilsdecoratorsnoisyandsleep.Each
Benchmarkshould also have a propertyscorerthat returns a subclass ofHPOScorer(see definition) with methods that transform the objective in regret.How is this so-called "scorer" used? Can you point me to an example in the old CBBO and DTLZ code https://github.com/deephyper/benchmark/tree/0.0.1? In the old code I see a metrics.py module that has regret functions but I don't see any examples of this being used. Also, what does "regret" mean in this context, is it supposed to be regression?
- pinned this issue
on Mar 24, 2025 I added the remaining C-BBO benchmarks. Can we start a pull request for the cbbo branch to review the changes before merging with the main branch?
The DTLZ benchmarks should be refactored to follow the new setup.
refactoring