How to make a raytracer. PART 1

This tutorial is not for the faint of heart, if you aren't willing to install python 3.11 and other python packages then ignore this topic.

Installation of python and needed packages

First we need to install python 3.11 this can be done through the Microsoft Store if you're using windows.

Next we need to install the required packages, this can be done through the python command line or vs code (I will be using vs code)

Make sure you have pip installed via these comands:

Windows: py -m ensurepip --default-pip

Unix/macOs: python3 -m ensurepip --default-pip

If that doesn't allow you to run:

Windows: py -m pip --version

Unix/macOs: python3 -m pip --version

Then securely download get-pip.py (Search on google) Then run:

Windows: py get-pip.py --prefix=/usr/local/

Unix/macOs: python3 get-pip.py --prefix=/usr/local/

And now finally we can get to installing the required packages.

Prefix each package name with this command:

Windows: py -m pip install

Example: py -m pip install "SomePackage"

Unix/macOs: python3 -m pip install

Example: python3 -m pip install "SomePackage"

List of packages:

flask
pillow

Creation of the python server

First we need to designate an area for our python script and our images so make a folder somewhere and then create a new file in it called "main.py", this folder will be where all our images are saved so make sure you dont lose it!

Then open main.py in a text editor or vs code.

(MAKE SURE WHEN YOU OPEN IT NOT TO SELECT "Always use this app to open .py files")

(Final code at bottom if you wish to copy paste)

Now we need to import our packages.

from flask import Flask, abort, request
from tkinter import * 
from PIL import Image, ImageTk
import json
import time
from decimal import Decimal

After this we need to setup our flask app and define some variables.

app = Flask(__name__)

requestTypes = [
    "clear",
    "init",
    "write",
    "save"
]
pixelrows = []
currentImage = Image.new('RGB',(0, 0))
recievedLines = 0
imageSizeY = 0
globalMessageLoop = 0
img_size_x = 0
img_size_y = 0

Next we define our main function which handles command processing and image processing.

@app.route('/', methods=['POST'])
def sendCommand():
    customRequestType = requestTypes[int(request.form['request_type'])]
    print("Received request:", customRequestType)
    
    global img_size_x
    global img_size_y
    
    if customRequestType == "init":
        print("New image")
        print("size x:",request.form['image_size_x'])
        print("size y:",request.form['image_size_y'])

        global currentImage

        img_size_x = int(request.form['image_size_x'])
        img_size_y = int(request.form['image_size_y'])

        currentImage = Image.new('RGB',(img_size_x , img_size_y))

        return "Pass"

    if customRequestType == "write":
        # print(request.form)
        decodedTable = json.loads(request.form["pixel_data"], parse_float=Decimal)
        # decodedImageDataTable = json.loads(decodedTable, parse_float=Decimal)
        # print(decodedImageDataTable)
        
        print("Processing pixel data")

        y = int(request.form["y_row"])-1
        for x in range(img_size_x):
            print("Processing pixel: ("+str(x+1)+", "+str(y+1)+")")
            pixelColor = decodedTable[x]

            currentImage.putpixel( (x, y), (int(float(pixelColor[0])), int(float(pixelColor[1])), int(float(pixelColor[2]))) )
        
        return "Pass"
    
    if customRequestType == "save":
        imageName = time.strftime("%Y %m %d - %H %M %S") + ".png"
        currentImage.save(imageName, "PNG")
        
        root = Tk.Toplevel()
        img = ImageTk.PhotoImage(currentImage)
        panel = Label(root, image = img)
        panel.pack(side = "bottom", fill = "both", expand = "yes")
        root.mainloop()
        
        return "Pass"

And then at the end we run our app.

if __name__ == "__main__":
    app.run()

Our final script should look like this:

from flask import Flask, abort, request
from tkinter import * 
from PIL import Image, ImageTk
import json
import time
from decimal import Decimal

app = Flask(__name__)

requestTypes = [
    "clear",
    "init",
    "write",
    "save"
]
pixelrows = []
currentImage = Image.new('RGB',(0, 0))
recievedLines = 0
imageSizeY = 0
globalMessageLoop = 0
img_size_x = 0
img_size_y = 0

@app.route('/', methods=['POST'])
def sendCommand():
    customRequestType = requestTypes[int(request.form['request_type'])]
    print("Received request:", customRequestType)
    
    global img_size_x
    global img_size_y
    
    if customRequestType == "init":
        print("New image")
        print("size x:",request.form['image_size_x'])
        print("size y:",request.form['image_size_y'])

        global currentImage

        img_size_x = int(request.form['image_size_x'])
        img_size_y = int(request.form['image_size_y'])

        currentImage = Image.new('RGB',(img_size_x , img_size_y))

        return "Pass"

    if customRequestType == "write":
       decodedTable = json.loads(request.form["pixel_data"], parse_float=Decimal)
                  
        print("Processing pixel data")

        y = int(request.form["y_row"])-1
        for x in range(img_size_x):
            print("Processing pixel: ("+str(x+1)+", "+str(y+1)+")")
            pixelColor = decodedTable[x]

            currentImage.putpixel( (x, y), (int(float(pixelColor[0])), int(float(pixelColor[1])), int(float(pixelColor[2]))) )
        
        return "Pass"
    
    if customRequestType == "save":
        imageName = time.strftime("%Y %m %d - %H %M %S") + ".png"
        currentImage.save(imageName, "PNG")
        
        root = Tk.Toplevel()
        img = ImageTk.PhotoImage(currentImage)
        panel = Label(root, image = img)
        panel.pack(side = "bottom", fill = "both", expand = "yes")
        root.mainloop()
        
        return "Pass"

if __name__ == "__main__":
    app.run()

Then press Ctrl+S to save or manually save.

And thats it for our python server! You should be able to leave this running as we write the raytracer.

To run it you should be able to just double click on the main.py file.

Raytracer server side

Now that we are onto lua it should be a fairly simple process

Make a server script in ServerScriptService, in this script is where we will handle sending the pixel data to our python server for it to process then save to the image.

Be sure to also make a RemoteEvent called requestEvent in ReplicatedStorage,

First we need to define some basic stuff like our data types and services.

local HttpService = game:GetService("HttpService")
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local RunService = game:GetService("RunService")

local stepped = RunService.Stepped
local data_types = {
	"clear",
	"init",
	"write",
	"save"
}

Next we need to make our recieve function:

local function receiveRequest(fields)
	local url = "http://localhost:5000"
	local data = ""

	for k, v in pairs(fields) do
		data = data .. ("&%s=%s"):format(
			HttpService:UrlEncode(k),
			HttpService:UrlEncode(v)
		)
	end
	data = data:sub(2) 

	local res = nil
	task.spawn(function()
		warn("sending: ".. data_types[fields["request_type"] + 1])
		local success, fail = pcall(function()
			res = HttpService:PostAsync(url, data, Enum.HttpContentType.ApplicationUrlEncoded, false)
		end)

		if not success and fail:find("1024") then
			--end_request_batch_size /= 2
			warn(fail)
		end
	end)

	stepped:Wait()

	repeat
		stepped:Wait()
	until res
	return "Ready"
end

After this we make a render function which deals with sending the pixel cache info

local function render(fields)
	local sizey = fields.imageSize.Y -- get the full image size on the y axis

	local data = {
		["request_type"] = 2;
		["pixel_data"] = nil;
		["y_row"] = 0
	}

	local ready = false

	for row = 1,sizey do -- loop from 1 to the size y
		task.spawn(function()
			data.y_row = row -- sets the y_row data to the row value
			data.pixel_data = HttpService:JSONEncode(fields.pixel_data[row]) -- gets the pixel data row and sets it

			local ans = receiveRequest(data) -- calls the python server
			if row == sizey then
				repeat
					stepped:Wait()
				until ans == "Ready"
				ready = true
			end
		end)
	end

	repeat
		stepped:Wait()
	until ready == true

	return ready
end

And then finally we put our event handler.

ReplicatedStorage.requestEvent.OnServerInvoke = function(_,fields)	
	local ans = nil
	if fields.request_type == 2 then
		ans = render(fields)
		return ans
	end

	ans = receiveRequest(fields)
	return ans
end

Now our final script should look like this:

local HttpService = game:GetService("HttpService")
local ReplicatedStorage = game:GetService("ReplicatedStorage")
local RunService = game:GetService("RunService")

local stepped = RunService.Stepped
local data_types = {
	"clear",
	"init",
	"write",
	"save"
}

local function receiveRequest(fields)
	local url = "http://localhost:5000"
	local data = ""

	for k, v in pairs(fields) do
		data = data .. ("&%s=%s"):format(
			HttpService:UrlEncode(k),
			HttpService:UrlEncode(v)
		)
	end
	data = data:sub(2) 

	local res = nil
	task.spawn(function()
		warn("sending: ".. data_types[fields["request_type"] + 1])
		local success, fail = pcall(function()
			res = HttpService:PostAsync(url, data, Enum.HttpContentType.ApplicationUrlEncoded, false)
		end)

		if not success and fail:find("1024") then
			--end_request_batch_size /= 2
			warn(fail)
		end
	end)

	stepped:Wait()

	repeat
		stepped:Wait()
	until res
	return "Ready"
end

local function render(fields)
	local sizey = fields.imageSize.Y -- get the full image size on the y axis

	local data = {
		["request_type"] = 2;
		["pixel_data"] = nil;
		["y_row"] = 0
	}

	local ready = false

	for row = 1,sizey do -- loop from 1 to the size y
		task.spawn(function()
			data.y_row = row -- sets the y_row data to the row value
			data.pixel_data = HttpService:JSONEncode(fields.pixel_data[row]) -- gets the pixel data row and sets it

			local ans = receiveRequest(data) -- calls the python server
			if row == sizey then
				repeat
					stepped:Wait()
				until ans == "Ready"
				ready = true
			end
		end)
	end

	repeat
		stepped:Wait()
	until ready == true

	return ready
end

ReplicatedStorage.requestEvent.OnServerInvoke = function(_,fields)	
	local ans = nil
	if fields.request_type == 2 then
		ans = render(fields)
		return ans
	end

	ans = receiveRequest(fields)
	return ans
end

Thats it for our server side, now lets move onto the client raytracer.

Setting up

Make a folder in workspace called "Trace", this is where all the things we are going to raytrace will go.

And make a part in workspace called "Sun", make sure this is anchored as this is what we will use to calculate shadows.

Make a new ScreenGui in StarterGui called "RenderResult",

Inside of this make a frame called "viewFrame" this is what we will use to select the area we raytrace.

Change the propertys of viewFrame to this:

BackgroundColor3 = 255,82,83

BorderColor3 = 149,48,48
BackgroundTransparency = 0.5
BorderSizePixel = 7
Size = 0,0,0,0

Next in ReplicatedStorage make a module called "Utils"

Inside that module put this code:

local module = {}

-- Our raycast function just to make the code a bit cleaner
function module.Raycast(Origin,Direction,MaxDistance,Allow)
	local params = RaycastParams.new()
	params.FilterType = Enum.RaycastFilterType.Whitelist
	params.FilterDescendantsInstances = Allow


	local Result = workspace:Raycast(Origin,Direction * MaxDistance,params)

	return Result
end

-- Our reflect function, if you would like to know more you can search up "Roblox reflect math"
function module.Reflect(Direction : Vector3 ,Normal : Vector3) : Vector3
	return Direction - (2 * Direction:Dot(Normal) * Normal)
end

return module

Next we need to create our client main, make a local script called "Main" and put it in StarterPlayerScripts.

Inside of this local script put this:

local ReplicatedStorage = game:GetService("ReplicatedStorage")
local Players = game:GetService("Players")
local RunService = game:GetService("RunService")

local Raytracer = require(script.Parent:WaitForChild("Raytracer"))

local Player = Players.LocalPlayer
local ViewFrame = Player:WaitForChild("PlayerGui",math.huge):WaitForChild("RenderResult",math.huge):WaitForChild("viewFrame",math.huge)
local Mouse = Player:GetMouse()

local function startSelection()
	local xx = 0

	repeat
		RunService.RenderStepped:Wait()
		ViewFrame.Position = UDim2.new(0,Mouse.X,0,Mouse.Y)

		Mouse.Button1Down:connect(function()
			if xx == 0 then
				xx = 1
			end
		end)
	until xx == 1

	warn("selection 2")

	repeat
		RunService.RenderStepped:Wait()
		local X = ViewFrame.Position.X.Offset
		local Y = ViewFrame.Position.Y.Offset

		local Xa = Mouse.X
		local Ya = Mouse.Y

		local Xb = Xa-X
		local Yb = Ya-Y

		ViewFrame.Size = UDim2.new(0,Xb,0,Yb)

		Mouse.Button1Down:connect(function()
			xx = 2
		end)
	until xx == 2

	--// flip frame to position-size instead of size-position

	local frame = ViewFrame
	local x,y = frame.AbsoluteSize.X,frame.AbsoluteSize.Y

	local x2,y2 = math.abs(x),math.abs(y)

	local posX,posY = frame.AbsolutePosition.X,frame.AbsolutePosition.Y
	local sizeX,sizeY = frame.AbsoluteSize.X,frame.AbsoluteSize.Y

	if x2 ~= x then
		posX -= x2
	end
	if y2 ~= y then
		posY -= y2
	end
	sizeX = x2
	sizeY = y2

	ViewFrame.Position = UDim2.new(0,posX,0,posY)
	ViewFrame.Size = UDim2.new(0,sizeX,0,sizeY)

	warn("finished selection")

	local startPos = Vector2.new(ViewFrame.AbsolutePosition.X,ViewFrame.AbsolutePosition.Y)
	local endPos = startPos + Vector2.new(ViewFrame.AbsoluteSize.X,ViewFrame.AbsoluteSize.Y)
	return startPos,endPos
end

local s,e = startSelection()
local SizeX = math.abs(s.X-e.X)
local SizeY = math.abs(s.Y-e.Y)

local tracer = Raytracer.init(SizeX,SizeY,s)
tracer:Raytrace()

Next we need to create the module where our raytracing code will go,

Make a module script called "Raytracer" and put it in StarterPlayerScripts, Then inside of that module script make another module script called "Settings"

Inside of settings put this table:

return {
	SkyColor = Color3.fromRGB(208, 241, 255),
	MaxDistance = 500,
	Samples = 20,
}

This will control our SkyColor the MaxDistance for our Raycasts and how many bounces a reflection can have.

Ive reached the 16k character limit so I have to make a part 2, you should be able to find it via searching "How to make a raytracer. PART 2".

LEAVE ANY ISSUES YOU HAVE ON PART 2

How to make a raytracer. PART 1

An indepth tutorial on how one could make a simple raytracer

by jake_4543

Installation of python and needed packages

Creation of the python server

Raytracer server side

Setting up

View in-game to comment, award, and more!