We noticed that when using a 8192 MB memory limit, Madagascar frequently
surpasses this limit. This is probably due to a difference in how the
memory consumption is measured in the benchmark runner and Madagascar.
For this reason, this commit raises the memory limit by a tolerance
margin of 1024 MB.
With this change, the benchmark runner looks for per-problem domain
files in the case that there is no single domain.pddl file. This is
necessary for a range of domains where there is a domain file for each
problem file.
Checking out branches with many files consumes increasing amounts of
time. For this reason, this commit introduces worktrees that represent
the individual branches in different directories, while still
maintaining only one copy of the .git directory.
Due to a mistake, all instances were tested for all configurations. This
commit checks whether a specific instance is actually required for the
tested configuration and skips it if not.
With this commit, the error output is required to exist in order for an
instance to be considered done. This is just a defensive measure to
ensure that no useful information can be missed.
This enforces all Git commands that are absolutely relevant for keeping
the benchmark running to succeed. If they don’t, an error is immediately
thrown. This doesn’t include fetching and pushing, because syncing the
data is still possible after downtimes, for example.
The Git history is sufficient for viewing past status messages.
Additionally, long status log files tend to take long to render. With
this commit, only the most recent 100 lines are kept, and this number
can be configured.
Patrick Lühne