代做Scalable Distributed Systems代写Java编程

日期：2024-03-24 08:46

Computer Sicence

Scalable Distributed Systems

Project #1 Single Server, Key-Value Store (TCP and UDP)

Assignment Overview

For this project, you will write a server program that will serve as a key value store. It will be set up to allow a single client to communicate with the server and perform three basic operations:     1) PUT (key, value)

2) GET (key)

3) DELETE (key)

A Hash Map could be used for setting up Key value stores.

For this project you will set up your server to be single-threaded and it only has to respond to a  single request at a time (e.g. it need not be multi-threaded – that will be part of Project #2). You must also use two distinct L4 communication protocols: UDP and TCP. What this means is that your client and server programs must use sockets (no RPC….yet, that’s project #2) and be

configurable such that you can dictate that client and server communicate using UDP for a given test run, but also be able to accomplish the same task using TCP. If you choose, you could have   two completely separate sets of applications, one that uses UDP and one that uses TCP or you      may combine them.

Your implementation may be written in Java. Your source code should be well-factored and well- commented. That means you should comment your code and appropriately split the project into    multiple functions and/or classes; for example, you might have a helper function/class to encode/ decode UDP packets for your protocol, or you might share some logic between this part of the

assignment and the TCP client/server in the next part.

The client must fulfill the following requirements:

• The client must take the following command line arguments, in the order listed:

o The hostname or IP address of the server (it must accept either).

o The port number of the server.

• The client should be robust to server failure by using a timeout mechanism to deal with an

unresponsive server; if it does not receive a response to a particular request, you should note it in a client log and send the remaining requests.

• You will have to design a simple protocol to communicate packet contents for the three request  types along with data passed along as part of the requests (e.g. keys, values, etc.) The client must be robust to malformed or unrequested datagram packets. If it receives such a datagram packet, it should report it in a human-readable way (e.g., “received unsolicited response acknowledging unknown PUT/GET/DELETE with an invalid KEY” - something to that effect to denote an receiving an erroneous request) to your server log.

• Every line the client prints to the client log should be time-stamped with the current system  time. You may format the time any way you like as long as your output maintains millisecond precision.

• You must have two instances of your client (or two separate clients):

o One that communicates with the server over TCP

o One that communicates with the server over UDP

The server must fulfill the following requirements:

• The server must take the following command line arguments, in the order listed:

• The port number it is to listen for datagram packets on.

• The server should run forever (until forcibly killed by an external signal, such as a Control-C, a kill, or pressing the “Stop” button in Eclipse).

• The server must display the requests received, and its responses, both in a human readable

fashion; that is, it must explicitly print to the server log that it received a query from a particular InetAddress and port number for a specific word, and then print to the log its response to     that query.

• The server must be robust to malformed datagram packets. If it receives a malformed datagram packet, it should report it in a human-readable way (e.g., “received malformed request of length  n from

:”) to the server log. Do not attempt to just print malformed datagram     packets to standard error verbatim; you won’t like the results.

• Every line the server prints to standard output or standard error must be time-stamped with the current system time (i.e., System.currentTimeMillis()). You may format the time any way you like as long as your output maintains millisecond precision.

• You must have.two instances of your server (or two separate servers):

o One that communicates with the server over TCP

o One that communicates with the server over UDP

Other notes:

You should use your client to pre-populate the Key-Value store with data and a set of keys. The    composition of the data is up to you in terms of what you want to store there. Once the key-value store is populated, your client must do at least five of each operation: 5 PUTs, 5 GETs, 5 DELETEs.

Evaluation

Your single-threaded Key-Value Store client and server will be evaluated on how well they  interoperate with each other and with the sample client and server provided, as well as their conformance to the requirements above.

Executive Summary

Part of your completed assignment submission should be an executive summary containing an     “Assignment overview” (1 paragraph, up to about 250 words) explaining what you understand to be the purpose and scope of the assignment and a “technical impression” (1–2 paragraphs, about  200–500 words) describing your experiences while carrying out the assignment. Provide a use     case (application) where you would apply this in practice. The assignment overview shows how   well you understand the assignment; the technical impression section helps to determine what parts of the assignment need clarification, improvement, etc., for the future.

Evaluation

The grade for your executive summary is based on the effort you put into the assignment overview and

technical impression. In general, if you put some effort into your writing, you will receive full credit for

your executive summary (provided that it is properly formatted and submitted as a plain text file).

Project Deliverables

The following items should be archived together, e.g., placed in a  .zip file or tarball file (*.tgz or *.tar.gz), and electronically submitted via the link is provided on the course Moodle page.

1. All novel Java source code files implementing the two client and two server programs, i.e., plus any additional support code.

2. Your executive summary.

Project Submissions Guidelines

## General guidelines

* Please spend some time to make a proper `ReadME` markdown file, explaining all the steps necessary to execute your source code.

* Do not hardcode IP address or port numbers, try to collect these configurable information from config file/env variables/cmd input args.

* Attach screenshots of your testing done on your local environment.

## Packaging the application

* We'll make use of [Docker](https://en.wikipedia.org/wiki/Docker_(software)) to package and distribute our applications as docker containers! Please spend some time understanding the

[basics](https://docs.docker.com/get-started/) of docker.

* Please install Docker desktop

### Sample configuration #### Project structure * Before we jump to packaging our application, make sure the source code follows a similar structure with `client` and `server` packages. ```bash src ├── Dockerfile ├── Project\ 1.iml ├── Project\ Submission\ Guidelines.md ├── Readme.md ├── client │   ├── AbstractClient.java │   ├── Client.java │   ├── ClientApp.java│   ├── ClientLogger.java │   ├── TCPClient.java │   └── UDPClient.java ├── deploy.sh ├── run_client.sh └── server ├── AbstractHandler.java ├── KeyValue.java ├── Response.java ├── ServerApp.java ├── ServerLogger.java ├── TCPHandler.java └── UDPHandler.java 2 directories, 19 files ``` * Compile the code using `javac server/*.java client/*.java` * server usage should then be similar to `java server.ServerApp ` * client usage should then be similar to `java client.ClientApp ` #### Dockerfile * Use any base image from [openJDK](https://hub.docker.com/_/ openjdk) based on Alpine(takes less memory) * You could use two separate Dockerfiles for client and server, or make use of multistage Dockerfile as shown below and target intermediate images ##### Example Dockerfile ```dockerfile FROM bellsoft/liberica-openjdk-alpine-musl:11 AS client-build COPY . /usr/src/myapp WORKDIR /usr/src/myapp RUN javac client/*.java FROM bellsoft/liberica-openjdk-alpine-musl:11 AS server-build COPY . /usr/src/myapp WORKDIR /usr/src/myapp RUN javac server/*.java # cmd to run server locally - java server.ServerApp 1111 5555 CMD ["java", "server.ServerApp", "1111", "5555"] ``` ##### 1. Build * Run `docker build -t --target server-build .`to build the server docker images. * This example should create a docker image named . * Entrypoint is defined for server using CMD, but not for client as we need to run it manually with interactive option to input our desired operations from the console. * Note: `bellsoft/liberica-openjdk-alpine-musl:11` is an alpine based image that works on M1 Apple Silicon chips. You could choose any default linux/windows based images from [openJDK] (https://hub.docker.com/_/openjdk). ##### 2. Running server * Run `docker run -p 1111:1111/tcp -p 5555:5555/udp --name ` * This should run your server image and expose & map the respective ports on the container * You can now test your server container, with a copy of local client application ##### 3. Running client * Build `docker build -t --target client-build .` * Run `docker run -it --rm --name java client.ClientApp localhost 1111 tcp` should run the client docker image on interactive mode * This can now be tested with your server running on localhost (not the docker container, yet) Note: Both server and client docker containers are not on the local host network, so they cannot communicate with each other, yet! But, if one of the programs is running on you local env, then they will be able to communicate! ##### 4. Custom network * To facilitate the docker containers to communicate with each other, we need to create a virtual network and attach both our containers to this virtual network * Run `docker network create ` to create a network * Attach the containers with `--network ` option while running your server or client containers * Example: `docker run -p 1111:1111/tcp -p 5555:5555/udp --name --network ` * Note: dockers attached to custom networks have default DNS as container name, hence we can use docker container name instead of virtual IP address or localhost.#### Scripting * We can automate all 4 steps above using shell scripts to avoid repeating frequently used commands ```shell PROJECT_NETWORK='project1-network' SERVER_IMAGE='project1-server-image' SERVER_CONTAINER='my-server' CLIENT_IMAGE='project1-client-image' CLIENT_CONTAINER='my-client' # clean up existing resources, if any echo "----------Cleaning up existing resources----------" docker container stop $SERVER_CONTAINER 2> /dev/null && docker container rm $SERVER_CONTAINER 2> /dev/null docker container stop $CLIENT_CONTAINER 2> /dev/null && docker container rm $CLIENT_CONTAINER 2> /dev/null docker network rm $PROJECT_NETWORK 2> /dev/null # only cleanup if [ "$1" == "cleanup-only" ] then exit fi # create a custom virtual network echo "----------creating a virtual network----------" docker network create $PROJECT_NETWORK # build the images from Dockerfile echo "----------Building images----------" docker build -t $CLIENT_IMAGE --target client-build . docker build -t $SERVER_IMAGE --target server-build . # run the image and open the required ports echo "----------Running sever app----------" docker run -d -p 1111:1111/tcp -p 5555:5555/udp --name $SERVER_CONTAINER --network $PROJECT_NETWORK $SERVER_IMAGE echo "----------watching logs from server----------" docker logs $SERVER_CONTAINER -f ``` * Above script. `deploy.sh` should help you build and deploy server images along with a custom network ```shell CLIENT_IMAGE='project1-client-image'PROJECT_NETWORK='project1-network' SERVER_CONTAINER='my-server' if [ $# -ne 3 ] then echo "Usage: ./run_client.sh " exit fi # run client docker container with cmd args docker run -it --rm --name "$1" \ --network $PROJECT_NETWORK $CLIENT_IMAGE \ java client.ClientApp $SERVER_CONTAINER "$2" "$3" # cmd to run client locally - java client.ClientApp localhost 1111 tcp ``` * `run_client.sh` script. above, should help you start a client container on the same network Note: Do not forget to change the permission of sh files to executable `chmod +x *.sh` ## Helpful tools and commands ### Tools * [iTerm2](https://iterm2.com/) - should help in managing multiple terminals, we'll be running upto 8 terminals in upcoming projects * [Oh my zsh](https://ohmyz.sh/) - helps in code completions and cool themes * add `plugins=(git docker docker-compose kubectl)` to `.zshrc` and restart * alternatives - [fish](http://fishshell.com/), [ohmybash] (https://github.com/ohmybash/oh-my-bash) * If you prefer GUI, official Docker Plugins on VS Code and IntelliJ or even Docker Desktop ### Docker commands * list running containers - `docker container ls` * list all containers - `docker container ls -a` * list all networks - `docker network ls` * inspect containers attached to a network - `docker network inspect ` * stop running container - `docker container stop `* delete container - `docker container rm ` * delete network - `docker network rm ` ## TL;DR * Use the Dockerfile provided to package your server and client apps as Docker containers. * Use the provided shell scripts to automate deployment and running of those containers. * You only need to modify last line of `Dockerfile` with your server args and last line of `run_client.sh` with your client args ### Feel free to use any of these files as is or modify according to your needs!