test2

.gitignore

+__pycache__/
+tensorboard/
+testing_tensorboard_DQN.zip
+testing_tensorboard.zip
\ No newline at end of file

.vscode/settings.json

+{
+    "python.analysis.extraPaths": [
+        "./car_custom_gym_env_package"
+    ]
+}
\ No newline at end of file

README.md

+# Custom OpenAI Gym Environment
+
+This is a custom OpenAI gym environment designed for the Future IoT course at Jyväskylä University of Applied Sciences.
+
+## Installation
+
+To install this environment, clone the repository and do the following:
+
+1. Navigate to the root folder with Anaconda Promt (Windows) or Terminal (MacOS/Linux) - this is the folder that contains the `requirements.txt` file.
+2. Create a new conda environment with `conda create --name <env_name> python=3.9.2`
+3. Activate the environment with `conda activate <env_name>`
+4. Run the command `pip install setuptools==65.5.0 pip==21`
+    - You might get an error that you need to run a different command to modify pip, simply copy the command that is given to you and run it.
+5. Install the other required packages with `pip install -r requirements.txt`
+
+## Usage
+
+To use the environment, you can import it into your own projects as shown in the [examples](./examples/). You can also modify the files in the [examples](./examples/) folder to suit your needs.
+
+An important thing to remember is that you must always check the [init](./car_custom_gym_env_package/car_custom_gym_env/__init__.py) file and uncomment the use case you want to use. For example, if you want to train your model you must uncomment the line:
+
+```python
+from car_custom_gym_env.train_env import *
+```
+
+And leave the other two imports commented out.
+
+**Note: If using the examples, please ensure that you are running them from the root folder. Do not navigate to the examples folder and run `python train.py` from there. Run `python ./examples/train.py` from theroot folder in stead. If using a code editor such as VS Code, simply check the terminal to see where the Python interpreter is running the code from if you see any `module not found` errors.**
+
+### Example
+
+#### Training and testing a model on your own computer
+
+After installing everything according to the instructions above:
+
+1. Open up the code in your favorite editor from the root folder, in this example we will use VS Code. The view in the explorer should have the following folders and files:
+    - .vscode
+    - car_custom_gym_env_package
+    - examples
+    - .gitignore
+    - assa2.gif
+    - README.md
+    - requirements.txt
+2. Ensure that you have the correct Python interpreter selected, you can do this by pressing `Ctrl + Shift + P` and typing `Python: Select Interpreter` and selecting the correct one from the list - it should be the Anaconda environment you created earlier.
+3. Navigate to the [init](./car_custom_gym_env_package/car_custom_gym_env/__init__.py) file and uncomment the line:
+
+```python
+from car_custom_gym_env.train_env import *
+```
+
+4. Run the [train.py](./examples/train.py) file by pressing the play button in the top right corner of VS Code, if all is done correctly you will see the car start moving around in the environment, trying to learn how to drive. The training will take a while if you have `total_timesteps` set to a large number, so be patient.
+5. After the training is done, a new .zip folder will be created in the root folder, this is the model that was trained. You can now test the model by uncommenting the following line from the [init](./car_custom_gym_env_package/car_custom_gym_env/__init__.py) file:
+
+```python
+from car_custom_gym_env.eval_graphic import *
+```
+
+Next, run the [eval.py](./examples/eval.py) file by pressing the play button of that file in the top right corner of VS Code. You should see the car driving around in the environment, this time it is using the model that was trained earlier. Ensure that it's loading the correct model by checking the following line in the [eval.py](./examples/eval.py) file:
+
+```python
+model = PPO.load("./model_name.zip")
+```
+
+6. When testing it on the Raspberry Pi, you can simply copy the model's .zip file to the Raspberry Pi and run the [eval.py](./examples/eval.py) file there, and uncomment the appropriate line in [init](./car_custom_gym_env_package/car_custom_gym_env/__init__.py). Ensure that you have the an Anaconda env on the Raspberry Pi created in the same way as on your own computer and that you have the correct packages installed. You can install the packages in the same way as before, but make sure to also install `RPi.GPIO` (tested using version 0.7.0).
+
+**Common issues/helpful links:**
+
+-   Installing Anaconda: [Link](https://docs.anaconda.com/free/anaconda/install/index.html)
+-   Stable Baselines3 v1.4.0 documentation: [Link](https://stable-baselines3.readthedocs.io/en/v1.4.0/)
+-   Problems with installation of `gym==0.19.0`: [Link](https://github.com/openai/gym/issues/3176)

car-custom-gym-env @ e5bdb3ed

-Subproject commit e5bdb3edd6a25cc4c25eb0988c59ab416bd5071b

car_custom_gym_env_package/car_custom_gym_env/__init__.py

+# Uncomment below line for training, comment other import
+#from car_custom_gym_env.train_env import *
+
+# Uncomment below line for evaluation with car, comment other import
+#from car_custom_gym_env.eval_env import *
+
+# Uncomment below line for evaluation with graphics, comment other import
+from car_custom_gym_env.eval_graphic import *
+from gym.envs.registration import register
+
+register(
+    id='car-env-custom-v1',
+    entry_point='car_custom_gym_env:CarEnv',
+    kwargs={'render_sim': False, 'n_steps': 1000}
+)

car_custom_gym_env_package/car_custom_gym_env/eval_env.py

+import gym
+from gym import spaces
+import numpy as np
+
+from time import sleep
+import keyboard
+import sys
+
+import RPi.GPIO as gpio
+
+gpio.setmode(gpio.BCM)
+
+gpio.setup(19, gpio.OUT) # Left motor
+gpio.setup(26, gpio.OUT) # Right motor
+
+gpio.setup(16, gpio.IN) # Right sensor
+gpio.setup(20, gpio.IN) # Middle sensor
+gpio.setup(21, gpio.IN) # Left sensor
+
+class CarEnv(gym.Env):
+    """
+    render_sim: (bool) Unimplemented - no simulation for this environment
+    n_steps: (int) number of time steps
+    """
+
+    def __init__(self, render_sim=False, n_steps=1000):
+        self.render_sim = render_sim
+        self.max_time_steps = n_steps
+        self.done = False
+        self.current_time_step = 0
+
+        self.action_space = spaces.Discrete(3)
+        self.observation_space = spaces.MultiBinary(3)
+    
+    def step(self, action):
+        if keyboard.is_pressed('q'):
+                sys.exit("Closed by user")
+
+        if action == 0:
+            turn_left()
+        elif action == 1:
+            turn_right()
+        else:
+            move_forward()
+        
+        obs=np.array([0, 0, 0])
+
+        if gpio.input(21)==1: obs[0]=1
+        if gpio.input(20)==1: obs[1]=1
+        if gpio.input(16)==1: obs[2]=1
+
+        reward=-200
+
+        if obs[0] == 1:
+            reward += 200
+        if obs[1] == 1:
+            reward += 1300
+            if obs[0] == 0 and obs[2] == 0:
+                reward += 800
+        if obs[2] == 1:
+            reward += 200    
+
+        self.current_time_step += 1
+        if self.current_time_step == self.max_time_steps:
+            self.done = True
+
+        return obs, reward, self.done, {}
+
+    def render(self, mode='human', close=False):        
+        print("Rendering...")
+        
+    def reset(self):
+        obs=np.array([0, 0, 0])
+
+        if gpio.input(21)==1: obs[0]=1
+        if gpio.input(20)==1: obs[1]=1
+        if gpio.input(16)==1: obs[2]=1
+
+        return obs
+
+    def close(self):
+        print("Closed")
+
+def turn_left():
+    gpio.output(26, 1)
+    sleep(0.5)
+    gpio.output(26, 0) # = about -10 degrees (estimate)
+    print("left")
+    
+def turn_right():
+    
+    gpio.output(19, 1)
+    sleep(0.5)
+    gpio.output(19, 0) # about +10 degrees (estimate)
+    print("right")
+    
+def move_forward():
+    gpio.output(26, 1)
+    gpio.output(19, 1)
+    sleep(0.5)
+    gpio.output(26, 0)
+    gpio.output(19, 0)
+    
+    print("forward")
\ No newline at end of file

car_custom_gym_env_package/car_custom_gym_env/eval_graphic.py

+import gym
+from gym import spaces
+from gym.utils import seeding
+import numpy as np
+import math
+import os
+
+from time import sleep
+from graphics import Line, Circle, Point, GraphWin, Image
+import keyboard
+import sys
+
+# Graphics setup
+angle = 75
+x2 = 50
+y2 = 350
+
+l=Line(Point(0, 0), Point(0, 0))
+l2=Line(Point(0, 0), Point(0, 0))
+l3=Line(Point(0, 0), Point(0, 0))
+l4=Line(Point(0, 0), Point(0, 0))
+l5=Line(Point(0, 0), Point(0, 0))
+l6=Line(Point(0, 0), Point(0, 0))
+l7=Line(Point(0, 0), Point(0, 0))
+l8=Line(Point(0, 0), Point(0, 0))
+
+c1=Circle(Point(0, 0), 7)
+c2=Circle(Point(0, 0), 7)
+c3=Circle(Point(0, 0), 7)
+
+paper = GraphWin(width=800, height=800)
+myImage = Image(Point(400,400), 'assa2.gif')
+myImage.draw(paper)
+
+class CarEnv(gym.Env):
+    """
+    render_sim: (bool) Unimplemented - simulation is always rendered
+    n_steps: (int) number of time steps
+    """
+
+    def __init__(self, render_sim=False, n_steps=1000):
+        self.render_sim = render_sim
+        self.max_time_steps = n_steps
+        self.done = False
+        self.current_time_step = 0
+
+        self.action_space = spaces.Discrete(3)
+        self.observation_space = spaces.MultiBinary(3)
+    
+    def step(self, action):
+        if keyboard.is_pressed('q'):
+                sys.exit("Closed by user")
+
+        if action == 0:
+            turn_left()
+        elif action == 1:
+            turn_right()
+        else:
+            move_forward()
+        
+        global x2, y2, angle, myImage
+    
+        length=50
+        length2=50/4
+        length3=50*2
+    
+        rad2=math.cos(math.radians(angle))
+        rad=math.sin(math.radians(angle))
+        
+        draw_car()
+
+        x3=int(x2 + length2*rad2 - length3*rad)
+        y3=int(y2 + length2*rad + length3*rad2)
+        if x3<1: x3=1
+        if x3>799: x3=799
+        if y3<1: y3=1
+        if y3>799: y3=799
+        red, green, blue = myImage.getPixel(x3, y3)
+        
+        x3=int(x2 - length2*rad2 - length3*rad)
+        y3=int(y2 - length2*rad + length3*rad2)
+        if x3<1: x3=1
+        if x3>799: x3=799
+        if y3<1: y3=1
+        if y3>799: y3=799
+        red2, green2, blue2 = myImage.getPixel(x3, y3)
+        
+        x3=int(x2 - length3*rad)
+        y3=int(y2 + length3*rad2)
+        if x3<1: x3=1
+        if x3>799: x3=799
+        if y3<1: y3=1
+        if y3>799: y3=799
+        red3, green3, blue3 = myImage.getPixel(x3, y3)
+
+        obs=np.array([0, 0, 0])
+        
+        if red==0 and green==0 and blue==0: obs[0]=1
+        if red2==0 and green2==0 and blue2==0: obs[2]=1
+        if red3==0 and green3==0 and blue3==0: obs[1]=1
+
+        reward=-200
+
+        if obs[0] == 1:
+            reward += 200
+        if obs[1] == 1:
+            reward += 1300
+            if obs[0] == 0 and obs[2] == 0:
+                reward += 800
+        if obs[2] == 1:
+            reward += 200
+
+        if x2<0:
+            x2=800
+        if x2>800:
+            x2=0
+        if y2<0:
+            y2=800
+        if y2>800:       
+            y2=0
+
+        self.current_time_step += 1
+        if self.current_time_step == self.max_time_steps:
+            self.done = True        
+
+        return obs, reward, self.done, {}
+
+    def render(self, mode='human', close=False):        
+        # print("Pois päältä...")
+        pass
+        
+    def reset(self):
+        return np.array([0, 0, 0])
+
+    def close(self):
+        # print("Heippa!")
+        pass
+
+def draw_car():
+    length=50
+    length2=50/4
+    length3=50*2
+    global angle, l, l2, l3, l4, l5, l6, x2, y2, c1, c2, c3
+ 
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+   
+    l.undraw()
+    l2.undraw()
+    l3.undraw()
+    l4.undraw()
+    l6.undraw()
+    
+    c1.undraw()
+    c2.undraw()
+    c3.undraw()
+    
+    l=Line(Point(x2, y2), Point(x2 + length*rad2, y2 + length*rad))
+    l2=Line(Point(x2, y2), Point(x2 - length*rad2, y2 - length*rad))
+    l3=Line(Point(x2 + length2*rad2, y2 + length2*rad), Point(x2 + length2*rad2 - length3*rad , y2 + length2*rad + length3*rad2))
+    l4=Line(Point(x2 - length2*rad2, y2 - length2*rad), Point(x2 - length2*rad2 - length3*rad , y2 - length2*rad + length3*rad2))
+    
+    l6=Line(Point(x2, y2), Point(x2 - length3*rad, y2 + length3*rad2))
+    
+    c1=Circle(Point(x2 + length2*rad2 - length3*rad , y2 + length2*rad + length3*rad2), 7).draw(paper)
+    c2=Circle(Point(x2 - length2*rad2 - length3*rad, y2 - length2*rad + length3*rad2), 7).draw(paper)
+    c3=Circle(Point(x2 - length3*rad, y2 + length3*rad2), 7).draw(paper)
+   
+    l.draw(paper)
+    l2.draw(paper)
+    l3.draw(paper)
+    l4.draw(paper)
+    l6.draw(paper)
+
+def turn_left():
+    global angle, x2, y2
+    
+    length=50
+    length2=50/4
+    length3=50*2
+    
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+    
+    x4=x2 + length*rad2
+    y4=y2 + length*rad
+    
+    angle=angle-1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360   
+        
+    if angle<90:
+        angle2=angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad2
+        y2=y4 - length*rad
+    elif angle>=90 and angle<180:
+        angle2=90-(angle-90)
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad2
+        y2=y4 - length*rad
+    elif angle>=180 and angle<270:
+        angle2=270-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad
+        y2=y4 + length*rad2
+    elif angle>=270:
+        angle2=360-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad2
+        y2=y4 + length*rad
+        
+    angle=angle-1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360
+    
+def turn_right():
+    global angle, x2, y2
+    
+    length=50
+    length2=50/4
+    length3=50*2
+    
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+    
+    x4=x2 - length*rad2
+    y4=y2 - length*rad
+    
+    angle=angle+1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360   
+        
+    if angle<90:
+        angle2=angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad2
+        y2=y4 + length*rad
+    elif angle>=90 and angle<180:
+        angle2=90-(angle-90)
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad2
+        y2=y4 + length*rad
+    elif angle>=180 and angle<270:
+        angle2=270-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad
+        y2=y4 - length*rad2
+    elif angle>=270:
+        angle2=360-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad2
+        y2=y4 - length*rad
+        
+    angle=angle+1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360
+    
+def move_forward():
+    global angle, x2, y2
+    
+    length=50 / 5
+    length2=50/4
+    length3=50*2
+    
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+    
+    x2=x2 - length*rad
+    y2=y2 + length*rad2
\ No newline at end of file

car_custom_gym_env_package/car_custom_gym_env/render.py

+# This file contains the code for rendering the environment for training purposes
+# The code is mostly the work of a previous student, but I have made minor changes to it
+
+import numpy as np
+import math
+from graphics import Line, Circle, Point, GraphWin, Image
+
+angle = 75
+x2 = 50
+y2 = 350
+
+l=Line(Point(0, 0), Point(0, 0))
+l2=Line(Point(0, 0), Point(0, 0))
+l3=Line(Point(0, 0), Point(0, 0))
+l4=Line(Point(0, 0), Point(0, 0))
+l5=Line(Point(0, 0), Point(0, 0))
+l6=Line(Point(0, 0), Point(0, 0))
+l7=Line(Point(0, 0), Point(0, 0))
+l8=Line(Point(0, 0), Point(0, 0))
+
+c1=Circle(Point(0, 0), 7)
+c2=Circle(Point(0, 0), 7)
+c3=Circle(Point(0, 0), 7)
+
+paper = GraphWin(width=800, height=800)
+myImage = Image(Point(400,400), 'assa2.gif')
+myImage.draw(paper)
+
+def draw_car():
+    length=50
+    length2=50/4
+    length3=50*2
+    global angle, l, l2, l3, l4, l5, l6, x2, y2, c1, c2, c3
+ 
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+   
+    l.undraw()
+    l2.undraw()
+    l3.undraw()
+    l4.undraw()
+    l6.undraw()
+    
+    c1.undraw()
+    c2.undraw()
+    c3.undraw()
+    
+    l=Line(Point(x2, y2), Point(x2 + length*rad2, y2 + length*rad))
+    l2=Line(Point(x2, y2), Point(x2 - length*rad2, y2 - length*rad))
+    l3=Line(Point(x2 + length2*rad2, y2 + length2*rad), Point(x2 + length2*rad2 - length3*rad , y2 + length2*rad + length3*rad2))
+    l4=Line(Point(x2 - length2*rad2, y2 - length2*rad), Point(x2 - length2*rad2 - length3*rad , y2 - length2*rad + length3*rad2))
+    
+    l6=Line(Point(x2, y2), Point(x2 - length3*rad, y2 + length3*rad2))
+    
+    c1=Circle(Point(x2 + length2*rad2 - length3*rad , y2 + length2*rad + length3*rad2), 7).draw(paper)
+    c2=Circle(Point(x2 - length2*rad2 - length3*rad, y2 - length2*rad + length3*rad2), 7).draw(paper)
+    c3=Circle(Point(x2 - length3*rad, y2 + length3*rad2), 7).draw(paper)
+   
+    l.draw(paper)
+    l2.draw(paper)
+    l3.draw(paper)
+    l4.draw(paper)
+    l6.draw(paper)
+
+def get_obs(): # Get sensor readings
+    global x2, y2, angle, myImage
+
+    obs=np.array([0, 0, 0])
+
+    length=50
+    length2=50/4
+    length3=50*2
+    
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+
+    x3=int(x2 + length2*rad2 - length3*rad)
+    y3=int(y2 + length2*rad + length3*rad2)
+    if x3<1: x3=1
+    if x3>799: x3=799
+    if y3<1: y3=1
+    if y3>799: y3=799
+    red, green, blue = myImage.getPixel(x3, y3)
+        
+    x3=int(x2 - length2*rad2 - length3*rad)
+    y3=int(y2 - length2*rad + length3*rad2)
+    if x3<1: x3=1
+    if x3>799: x3=799
+    if y3<1: y3=1
+    if y3>799: y3=799
+    red2, green2, blue2 = myImage.getPixel(x3, y3)
+        
+    x3=int(x2 - length3*rad)
+    y3=int(y2 + length3*rad2)
+    if x3<1: x3=1
+    if x3>799: x3=799
+    if y3<1: y3=1
+    if y3>799: y3=799
+    red3, green3, blue3 = myImage.getPixel(x3, y3) # Assuming that this is the middle sensor
+        
+    if red==0 and green==0 and blue==0: obs[0]=1
+    if red2==0 and green2==0 and blue2==0: obs[2]=1
+    if red3==0 and green3==0 and blue3==0: obs[1]=1 # Assuming that this is the middle sensor
+
+    return obs
+
+def center_car(): # Center car if it goes outside the screen
+    global x2, y2
+
+    if x2<0:
+        x2=800
+    if x2>800:
+        x2=0
+    if y2<0:
+        y2=800
+    if y2>800:       
+        y2=0
+
+def turn_left():
+    global angle, x2, y2
+    
+    length=50
+    length2=50/4
+    length3=50*2
+    
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+    
+    x4=x2 + length*rad2
+    y4=y2 + length*rad
+    
+    angle=angle-1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360   
+        
+    if angle<90:
+        angle2=angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad2
+        y2=y4 - length*rad
+    elif angle>=90 and angle<180:
+        angle2=90-(angle-90)
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad2
+        y2=y4 - length*rad
+    elif angle>=180 and angle<270:
+        angle2=270-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad
+        y2=y4 + length*rad2
+    elif angle>=270:
+        angle2=360-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad2
+        y2=y4 + length*rad
+        
+    angle=angle-1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360
+    
+def turn_right():
+    global angle, x2, y2
+    
+    length=50
+    length2=50/4
+    length3=50*2
+    
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+    
+    x4=x2 - length*rad2
+    y4=y2 - length*rad
+    
+    angle=angle+1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360   
+        
+    if angle<90:
+        angle2=angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad2
+        y2=y4 + length*rad
+    elif angle>=90 and angle<180:
+        angle2=90-(angle-90)
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad2
+        y2=y4 + length*rad
+    elif angle>=180 and angle<270:
+        angle2=270-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 - length*rad
+        y2=y4 - length*rad2
+    elif angle>=270:
+        angle2=360-angle
+        rad2=math.cos(math.radians(angle2))
+        rad=math.sin(math.radians(angle2))
+        x2=x4 + length*rad2
+        y2=y4 - length*rad
+        
+    angle=angle+1
+
+    if angle<0:
+        angle=360+angle
+    elif angle>360:
+        angle=angle-360
+
+def move_forward():
+    global angle, x2, y2
+    
+    length=50 / 5
+    length2=50/4
+    length3=50*2
+    
+    rad2=math.cos(math.radians(angle))
+    rad=math.sin(math.radians(angle))
+    
+    x2=x2 - length*rad
+    y2=y2 + length*rad2
\ No newline at end of file

car_custom_gym_env_package/car_custom_gym_env/train_env.py

+import gym
+from gym import spaces
+import numpy as np
+
+from graphics import *
+import keyboard
+import sys
+
+sys.path.append('./car_custom_gym_env_package/car_custom_gym_env')
+from render import *
+
+class CarEnv(gym.Env):
+    """
+    render_sim: (bool) Unimplemented - simulation is always rendered
+    n_steps: (int) number of time steps
+    """
+
+    def __init__(self, render_sim=False, n_steps=1000):
+        self.render_sim = render_sim
+        self.max_time_steps = n_steps      
+        self.done = False
+        self.current_time_step = 0
+
+        self.action_space = spaces.Discrete(3)
+        self.observation_space = spaces.MultiBinary(3)
+    
+    def step(self, action):
+        if keyboard.is_pressed('q'):
+                sys.exit("Closed by user")
+
+        if action == 0:
+            turn_left()
+        elif action == 1:
+            turn_right()
+        elif action == 2:
+            move_forward()
+        
+        draw_car()
+
+        obs = get_obs()
+
+        reward=-200
+
+        if obs[0] == 1:
+            reward += 200
+        if obs[1] == 1:
+            reward += 1300
+            if obs[0] == 0 and obs[2] == 0:
+                reward += 800
+        if obs[2] == 1:
+            reward += 200
+
+        center_car()
+
+        self.current_time_step += 1
+        if self.current_time_step == self.max_time_steps:
+            self.done = True
+
+        return obs, reward, self.done, {}
+
+    def render(self, mode='human', close=False):        
+        # Rendering is done in the step function
+        pass
+        
+    def reset(self):
+        obs = get_obs()
+        return obs
+
+    def close(self):
+        # Closing is done in the step function
+        pass
\ No newline at end of file

examples/eval.py

+from stable_baselines3 import PPO
+from stable_baselines3 import DQN
+import gym
+import time
+import sys
+import os
+
+sys.path.insert(0, './car_custom_gym_env_package') # train.py + eval.py
+import car_custom_gym_env
+
+continuous_mode = True # If True, after completing one episode the next one will start automatically
+random_action = False # If True, the agent will take actions randomly
+render_sim = True # If True, a graphic is generated
+
+env = gym.make('car-env-custom-v1', render_sim=render_sim, n_steps=1000)
+model = PPO.load("./model_name.zip")
+model.set_env(env)
+
+random_seed = int(time.time())
+model.set_random_seed(random_seed)
+
+obs = env.reset()
+
+# print("Starting program in 30 seconds...")
+# time.sleep(30)
+
+try:
+    while True:
+        if render_sim:
+            env.render()
+
+        if random_action:
+            action = env.action_space.sample()
+        else:
+            action, _states = model.predict(obs)
+
+        obs, reward, done, info = env.step(action)
+
+        if done is True:
+            if continuous_mode is True:
+                state = env.reset()
+            else:
+                break
+
+finally:
+    env.close()

examples/train.py

+from stable_baselines3 import PPO
+from stable_baselines3 import DQN
+import gym
+import sys
+
+sys.path.append('./car_custom_gym_env_package')
+import car_custom_gym_env
+
+env = gym.make('car-env-custom-v1', render_sim=False, n_steps=1000)
+
+# For Tensorboard logging, add a tensorboard_log argument to the PPO() method
+model = PPO("MlpPolicy", env, verbose=1)
+
+model.learn(total_timesteps=600000)
+model.save('model_name')
\ No newline at end of file

requirements.txt

+gym==0.19.0
+graphics.py==5.0.1.post1
+keyboard==0.13.5
+pygame==2.4.0
+pymunk==6.4.0
+stable-baselines3==1.4.0
+numpy==1.24.3
+tensorboard==2.14.0
\ No newline at end of file