MADWeb

MADWeb 2025 - Call for Artifacts

Before submitting your artifact, please check the information and submission guidelines below.

Important Dates

Submission page: https://madweb25.hotcrp.com/

Evaluation Process

Authors are invited to submit artifacts soon after receiving the paper notification. At least one contact author must be reachable and respond to questions in a timely manner during the entire evaluation period to allow round trip communications between the evaluators and the authors. At submission time, authors choose which badges they want to be evaluated for. Evaluators will evaluate each artifact using the instructions provided by the authors. Evaluators will communicate anonymously with authors through HotCRP to resolve minor issues and ask clarifying questions.

Artifact Badges

Authors can request their artifact to be evaluated towards one, two, or all of the following badges:

Artifact Details and Requirements

Artifacts can be, e.g., software, datasets, models, test suites, or mechanized proofs. Paper proofs are not accepted, as evaluators lack the time and often the expertise to carefully review them. Physical objects, such as specialized computer hardware, are also not accepted, due to the difficulty of making them available to evaluators.

To ensure that the evaluation is practical, the proposed experiments should take at most 1 day and run on a commodity desktop machine. When the paper’s research involves longer durations, the authors should design scaled-down experiments and properly justify how those can still significantly support the paper’s analyses. A commodity desktop machine is defined as one with an x86-64 CPU with 8 cores and 16 GB of RAM running a recent Linux or Windows operating system and software obtainable free of charge. If this requirement cannot be met, the authors should make arrangements to provide (e.g., via SSH to their own infrastructure, or renting it from providers) anonymous access to the evaluators. Hardware and software requirements must be clearly stated in the README file of the artifact. If you intend to provide access to special hardware, you must communicate this upon submission.

Artifact evaluation is single-blind. Each evaluator will independently test and review their assigned submissions. To maintain the anonymity of evaluators, artifact authors should not embed analytics or other tracking tools in any websites for their artifacts for the duration of the evaluation period. In cases where tracking is unavoidable, authors must notify the PC chairs in advance so that evaluators can take adequate safeguards.

Submitting an artifact does not give reviewers permission to make its contents public or to retain any part of it after evaluation. Thus, authors are free to include proprietary models, data files, or code in artifacts. Participating in the artifact evaluation process does not require the public release of artifacts, though it is highly encouraged.

Artifact Preparation

Artifacts must be packaged to ease evaluation and use, including instructions for the evaluators. Packaging is not only about evaluation, but also about future use of the artifact by other researchers who may want to build on top of it or use it as a baseline.

Some artifacts may attempt to perform malicious or destructive operations by design. Such cases should be explicitly flagged in detail at submission time so that evaluators can take appropriate precautions before installing and running these artifacts.

Instructions

Due to the lightweight nature of the evaluation process, the artifact appendix is not required. Instead, authors should provide a roadmap for evaluators as a new message on HotCRP while submitting the artifact. This message should include a description of the hardware, software, and other configuration requirements, and should enumerate the list of major claims made by the paper that can be reproduced through the artifact alongside the badge(s) the authors are aiming for. Authors should also provide a link to the artifact package, or upload the artifact package directly to HotCRP if the artifact cannot be made publicly available.

Nevertheless, an artifact package must include an exhaustive README document containing adequate instructions and other documentation to present and explain the nature and functionality of the artifact and, if applicable, to conduct the required experiments for result reproduction.

Artifact Packaging

Authors should consider one of the following methods to package the software components of their artifacts. Then, a link to access the package should be provided on HotCRP. The package can be uploaded directly to HotCRP if the available badge is not requested and the authors do not wish to publicly share their artifacts:

Resources

The following materials may be useful when preparing an artifact:

Acknowledgments

This page is adapted from the Call for Artifacts page of NDSS 2025.

madwebwork.bsky.social
infosec.exchange/@madwebwork
@madwebwork

MADWeb 2025, in cooperation with the NDSS Symposium