AutoGrader

Design

The AutoGrader uses a React frontend with a Typescript / node backend, using express for the server. There are two main objects passed between the front- and backend: User and Submission. To avoid the complexity of a shared module, these files are duplicated across the front- and backend, so if you change one, change the other.

Frontend Components

Path	Function
/grade
/profile
/submissions

Admin Features

1. Impersonation

   Admin users can impersonate any student, using the app as if they were that student. They can input a netid, name or github username in the student box and the grader will impersonate the first match to any one of these, including partial matches. Note that there can be students with the same name, so if there is any conflict, use the netid or github username.

2. Stats

   There is a stats component that gives a breakdown of submissions for each deliverable.

3. Admin Page

   Allows the admin to disable submissions, list admin user, add an admin, clear the database (drops all non-admin data) and restore the database from the most recent backup.

Server Endpoints

Unauthenticated

Endpoint	Function
/metadata.xml
/login
/assert
/logout
/admin
/report

Authenticated Non-Admin

Endpoint	Function
/user
/update
/grade

Admin Endpoints

Endpoint	Function
/impersonate
/stats
/toggle-submissions
/submissions-enabled
/admin/list
/admin/add
/admin/remove
/drop-data
/restore-data

Deployment

This app is hosted on an AWS EC2 server at cs329.click, and deployed using a shell script with a private key. The following command executes the script with the necessary parameters.

# -k keyfile -h hostname -s service
sh deploy.sh -k cs329.pem -h cs329.click -s grade

The script requires you to have a production configuration file named config.prod.js (which is ignored by git). To make one:

1. Copy the example config.ts into the backend folder.
2. cd backend and run npm run build and look at the config.js file it produces in the dist directory.
3. Copy this file into the root of the project (where deploy.sh is found), rename to config.prod.js and change the values to be those needed for the production autograder to run. These credentials can be found in the class AWS secrets repository (ask Professor Jensen for details).

The EC2 server has a local instance of MySQL running to which the autograder can connect.

The running service is managed on the server using pm2.

Example Configuration

The service configuration file must exist in the root of the backend directory. This is an example backend/config.ts file. Since the file contains credentials, it should never be committed.

export const config = {
  app: {
    // where the autograder is deployed
    hostname: 'https://cs329.cs.byu.edu',
  },
  db: {
    // credentials for db connection
    connection: {
      host: 'localhost',
      user: 'admin',
      password: 'blahblahblah',
      database: 'autograder',
      connectTimeout: 60000,
    },
  },
  canvas: {
    // api key from TA account for submitting grades
    token: '0000~1234567812345678123456781234567812345678',
    // the course number should be updated each semester
    base_url: `https://byu.instructure.com/api/v1/courses/26459`,
  },
  pizza_factory: {
    url: 'https://pizza-factory.cs329.click',
    // needed to add users as vendors, get tokens for them, and cause chaos
    authtoken: 'token',
  },
  github: {
    // access token for byucs329ta, used to trigger workflows
    personal_access_token: 'ghp_12345678_12345678',
  },
  logging: {
    source: 'autograder',
    url: 'https://logs-prod-006.grafana.net/loki/api/v1/push',
    userId: '123456',
    apiKey: 'glc_eyJxOjoiMCE...Q==',
  },
};

Authentication

This app uses the SAML protocol for authentication with BYU CAS as the service provider, using the saml2-js library to abstract most of the implementation. The certificates (byu.crt and sp.crt) for this are kept in the certs directory and are non-confidential. The sp.key file should not be publicly visible, hence it is created in the deployment script. The code for authentication is found in service.ts.

Local Development

The back and front end are run separately when running locally. Make sure you have a local instance of MySQL running and provide a config.ts file in the backend folder with correct credentials.

To run the backend:

cd backend && npm run dev

Run the frontend similarly:

cd frontend && npm run dev

Data

In order to have sample data to play with:

1. Create a SQL dump of the class database on the EC2 server

   1. SSH into the server
   2. Run the following:

   mysqldump -u <admin_user> -p autograder > dump.sql

2. Copy the SQL dump to your local environment

scp -i <path/to/private_key> ubuntu@cs329.click:/home/ubuntu/dump.sql <path/to/local/destination>

1. Populate your local database with the dumped data

mysql -u admin -p autograder < <path/to/dump.sql>

Authenticating Locally

Because localhost is not authorized to use CAS with SAML, you cannot log in this way when running the application locally. Instead, there is a page at the /admin-login path through which you can log in. The service maintains a separate admin table in which it keeps the netId and bcrypt encrypted password of admin users. You will need to add yourself to the local db to log in this way. To get the bcrypt version of your password to add to the db, you can run the following commands in the backend folder:

node
const bcrypt = require('bcrypt');
await bcrypt.hash('mypassword', 10);

Observability

The grader logs to an Autograder dashboard at https://byucs329ta.grafana.net/. You can log into this dashboard with the byucs329ta github account.