Submitting your work

Submission of this project consists of:

1. Writing a design doc (due Monday April 22),
2. Running your code against the official evaluation server (due Monday May 6),
3. Submitting the received submit token on Gradescope (due Monday May 6),
4. Doing a final presentation with a TA (early RRR week).

This guide will go over Steps 2 and 3.

Getting a group token

Your group has a unique group token which you will use to communicate with our API.

When the evaluation rescues are released, we will also release a Gradescope assignment where you can get a group token. Once you submit the group token assignment, your group is fixed and CANNOT be changed, so only do this when you are sure.

You must submit a file group.txt to the assignement. It should have the following format:

Group Declaration
Member 1 Name
Member 2 Name
Member 3 Name

The group declaration is shown in the example image below. You must copy it exactly.

If your registration was successful, you should get a response like:

{"token": "0LT4uIlNV9zKoYZzaBreaT9fuEc24MZfa8BojjN891Nb1zI5Mg="}

Create a file in the root of your client repository called group_token (exact same case and underscore). Put your group token in that file. For example, a group_token file might look like:

0LT4uIlNV9zKoYZzaBreaT9fuEc24MZfa8BojjN891Nb1zI5Mg=

This file is not kept in version control, so each group member should create the file on their own local copies. Do not share your group token with any other group.

Evaluating

Check the Policies page before continuing.

Evaluation this semester is slightly different. We have generated a number of fixed instances on our evaluation server. To get the most accurate score, all groups must run against all inputs.

There are 600 evaluation inputs total: 200 for each city. Each city is associated with a number of graphs (located in the graphs/ directory in the skeleton). Each graph will have some number of pre-generated inputs. It is your goal to do as well as possible against these inputs. Thus, we highly recommend starting evaluation only after you are confident in your solver.

To protect against server-side issues and connectivity issues, we will be dropping your lowest 72 scores.

You may only run against 24 inputs at a time. Each time you submit, you will be expected to type a short (fixed) statement showing your understanding that once an instance has been performed, you cannot rerun that instance, and you only have x inputs remaining.

Note: we are limiting the number of rescues performed each time to protect against accidental submission. We will not be giving any group extra instances.

Simply add --submit anywhere to the command you use to run your solver. For example, if you normally run python client.py --solver my_solver to test, you would run python client.py --solver my_solver --submit instead. A confirmation message should appear, and after entering the given phrase, your solver will run exactly 24 inputs.

Submit Token

After evaluating at least once, submit tokens will be found in the subdirectory submit_token. These contain your scores, and will be used as a backup in case we have server trouble.

Please upload your most recent one to Gradescope. You are permitted to re-upload. The most recent one can be found either by sorting by date modified, or sorting by name.