SIMULATION / FREE / MIT EXAMPLE SOURCE
Synapse
See a network learn
A real 2 → 3 → 1 network learns XOR while its weights, activations, loss curve, and decision field change live.
Take it for a spin.
Space pauses training, R resets weights, and One batch performs four gradient updates. Choose an input pair to inspect its activations.
Extract the Windows download to a writable folder and open neural-graph.exe. Escape closes it. Database examples keep their local data in the working folder.
See how it works.
Hidden layers, softsign activation, backpropagation, squared-error gradients, and nonlinear decision boundaries.
Small deterministic CPU training. The separate first-steps flying-dot lesson demonstrates GPU compute training.
READ / CHANGE / UNDERSTAND
The whole example is yours.
These are the exact files in the source download. Start with the model, then follow the window’s event loop. Comments explain the decisions.
// A 2 -> 3 -> 1 network learns XOR. Unlike a single linear neuron,
// the hidden layer can bend the decision boundary into two separate regions.
// Softsign is a smooth activation without an exponential dependency.
Softsign(float x) returns float
{
return x/(1.0+Math.AbsFloat(x))
}
Slope(float activation) returns float
{
float remainder=1.0-Math.AbsFloat(activation)
return remainder*remainder
}
class Network
{
list of float weights=[]
list of float hidden=[0.0,0.0,0.0]
float output=0
int steps=0
Network()
{
this.Reset()
}
Reset()
{
this.weights=[0.4,-0.6,0.2,-0.3,0.7,-0.4,0.8,0.5,0.1,0.6,-0.8,0.3,0.0]
this.steps=0
}
Predict(float x,float y) returns float
{
float sum=this.weights[12]
for j from 0 < 3
{
this.hidden[j]=Softsign(this.weights[j*3]*x+this.weights[j*3+1]*y+this.weights[j*3+2])
sum=sum+this.hidden[j]*this.weights[9+j]
}
this.output=Softsign(sum)
return this.output
}
Train(float x,float y,float target,float rate)
{
float prediction=this.Predict(x,y)
// For L = 1/2 (prediction-target)^2, the chain rule multiplies
// output error by the activation derivative, then each upstream weight.
float delta=(prediction-target)*Slope(prediction)
for j from 0 < 3
{
float upstream=delta*this.weights[9+j]*Slope(this.hidden[j])
// Calculate the hidden gradient before changing the outgoing weight.
this.weights[j*3]=this.weights[j*3]-rate*upstream*x
this.weights[j*3+1]=this.weights[j*3+1]-rate*upstream*y
this.weights[j*3+2]=this.weights[j*3+2]-rate*upstream
this.weights[9+j]=this.weights[9+j]-rate*delta*this.hidden[j]
}
this.weights[12]=this.weights[12]-rate*delta
this.steps=this.steps+1
}
Loss() returns float
{
float total=0
for sample from 0 < 4
{
float x=Math.IntToFloat(sample/2)*2.0-1.0
float y=Math.IntToFloat(sample-sample/2*2)*2.0-1.0
float target=-1
if x!=y
{
target=1
}
float difference=this.Predict(x,y)-target
total=total+difference*difference*0.125
}
return total
}
Batch()
{
for sample from 0 < 4
{
float x=Math.IntToFloat(sample/2)*2.0-1.0
float y=Math.IntToFloat(sample-sample/2*2)*2.0-1.0
float target=-1
if x!=y
{
target=1
}
this.Train(x,y,target,0.15)
}
}
}
Wire(int x,int y,int tx,int ty,float weight)
{
int r=103
int g=213
int b=183
if weight<0
{
r=229
g=124
b=168
}
int thickness=1+Math.RoundToInt(Math.MinFloat(Math.AbsFloat(weight),4.0))
DrawLineRgba(x,y,tx,ty,thickness,r,g,b,150)
}