Grid

Meta

I'm sure no one actually cares, but I apologize for the delay in this week's post. I started a new job this week, so I've been a little drained from getting adjusted to that.

Grid

Last week I discussed the next functionality to tackle. I concluded that implementing a grid would be a good easy-win to allow myself more of a fun/work balance. The grid basically took me the weekend to get right:

Actually, before I implemented the grid I finally closed out my story for selecting lines; what I had worked on before was individual vertices, but that functionality laid the groundwork for selecting line, triangle, and other types of selection handles.

I've implemented a grid a few times before. It's been tricky every time, but it basically boils down to a couple of things:

1. Knowing how many lines in each direction to draw
2. Knowing where to draw them

So, from a technical standpoint it actually boils down to one core thing:

1. Knowing where in (literal game) world the screen extents (left, right, bottom, and top edges) are.

Knowing the above with a given grid spacing (the units between the lines) gives us enough information to know exactly what lines to draw and where. To determine how many lines to draw, all is needed is to take the screen size and divide it by the grid spacing:

• verticalLines = (screenWidth / gridSpacing)
• horizontalLines = (screenHeight / gridSpacing)

The vertical lines are the lines positioned across the screen, so each one will have the same y values with varying x values. Conversely, the horizontal lines are the lines positioned down (or up, as my code actually does) the screen, so each one will have the same x values with varying y values.

Those x and y values can be determined from the Camera object, as it contains screen extent values that are updated every time the camera is moved (or zoomed, as I'll get to later). The camera is configured with the screen resolution and assumes its own position as the lower-left corner of the screen, making those calculations very simple:

• left = cameraPosition.x
• right = (cameraPosition.x + screenWidth)
• bottom = cameraPosition.y
• top = (cameraPosition.y + screenHeight)

These values are also used in the vertex shader to make objects actually move across the screen with the camera (so they don't have to manage that "visual state" manually).

To summarize up to this point:

1. The camera contains a position and manages the screen extents (the left, right, bottom, and top edges of the screen positioned in the game world) using the pre-configured screen size.
2. The number of vertical lines on the screen is determined by the screen width divided by the grid spacing.
3. The number of horizontal lines on the screen is determined by the screen height divided by the grid spacing.
4. Objects' positions on the screen are modified in the vertex shader to reflect camera movement so their "visual positions" don't have to be managed manually.

Consider the example of an unmoved camera with a screen size of 100x100 and a grid spacing of 10:

• The camera position would be (0, 0).
• The screen extents (left, right, bottom, top) would be: (0, 100, 0, 100).
• The number of vertical lines would be: 10 (100 / 10).
• The number of horizontal lines would be: 10 (100 / 10).

All vertical lines will start at the y-value of 0 and end at 100, but what are the x-values for them? Intuitively, we can see that there should be vertical lines at: 0, 10, 20, 30, 40, 50, 60, 70, 80, 90, 100. What if we start at the first x-value of screenExtents.left, and then incrementally add the gridSpacing for verticalLines count of lines?

for (i = 0, x = screenExtents.left; i < verticalLines; i++, x += gridSpacing) { verticalLine at x; }

That would give us vertical lines at: 0, 10, 20, 30, 40, 50, 60, 70, 80, 90

What about the one at 100? Well, it seems we needed to account for 0 in our vertical line count (which you might've also noticed above), making verticalLines = (screenWidth / gridSpacing) + 1

The same pattern would follow for horizontal lines.

Now consider the example of a camera moved to (-5, -15) with the same screen size of 100x100 and grid spacing of 10:

• The camera position would be (-5, -15).
• The screen extents (left, right, bottom, top) would be: (-5, 95, -15, 85).
• The number of vertical and horizontal lines would still be 10 each.

Using the same for-loop above, we would get vertical lines at: -5, 15, 25, 35, 45, 55, 65, 76, 85, 95, 105

(Notice that the one at 105 would be outside the visible extents of the screen, but that's okay because we're talking about two triangles in the end.)

Keeping in mind that the left edge of the screen is at -5 in the world,  it's evident that the grid moves with the camera--which is not what we want.

The remedy to this is actually a simple adjustment to our calculation:

verticalLineStart = floor(screenExtents.left / gridSpacing) * gridSpacing

or applied:

verticalLineStart = floor(-5 / 10) * 10 = floor(-0.5) * 10 = -1 * 10 = -10

And then modify our for-loop to:

for (i = 0, x = verticalLineStart; i < verticalLines; i++, x += gridSpacing) { verticalLine at x; }

which gives us vertical lines at: -10, 0, 10, 20, 30, 40, 50, 60, 70, 80, 90

The horizontal lines would follow the exact same formula, substituting the verticalLines with horizontalLines, screenExtents.left with screenExtents.bottom, and making horizontalLines at y instead of verticalLines at x.

Selection Algorithm, Unit Testing, Fatigue, Next Steps

Writing tools is deceptively difficult. A few mouse listeners, getting the right data structures down for the stuff the tools will edit, maybe some file IO utilities... no biggie... right? I can tell you the most difficult, tedious aspect so far has been getting the handling of mouse gestures solid.

What mouse gestures do I even want to support, and for what types of editing? What's being edited? These may seem obvious when thinking about any kind of geometry editor: Create meshes by drawing lines and triangles and then edit those meshes by dragging the individual vertices, edges, and entire triangles around. Shift+click should probably be a thing so multiple things can be edited at once, delete should probably eventually delete stuff, etc. There are definitely other actions that will need to be supported--especially so when taking into account all of the non-mesh things this system needs to support.

So, it's kind of clear what the editor should be like, but it's also very NOT clear. The truth is, a lot of stuff is still very fuzzy in my mind, including what functionality I even want for the first stages of actually building a game. A roadmap would help, but maintaining that is almost a full-time job on its own, and I'm wearing myself down a bit. I have my triangle though, so I'll start from there.

JavaScript gives you a number of very useful event handlers out of the box. Assuming a Chrome-based platform, they all even "just work." Well, kind of. The mouse drag events seem to only fire for working with HTML elements--not within elements like our WebGL canvas. That means drag gestures need to be handled manually, which is where a lot of the difficulty comes in. To edit my triangle, what structures and gestures exactly do I need to handle? Well, I've chosen the phrase Selection Handle to be the thing the mouse can interact with. (Selection) Handles are used to normalize what the mouse interacts with to simplify the input and mouse systems, leaving the specifics of how something is edited to the specific types of handles.

Given the dissemination of the correct list of handles from the active scene (only for the triangle so far), let's run through some scenarios:

1. Handles exist of a user-customizable size. Handles on the screen should dynamically change in size as the user adjusts it.
2. Non-selected or highlighted handles are dynamically rendered with a user-customizable color.
3. Mouse hover over handles alters their render color to another user-customizable (highlight) color.
4. Clicking a handle selects it.
5. Shift-clicking a handle toggles it as selected or unselected.
6. Clicking on no handle deselects all handles.
7. Shift+clicking+dragging creates a selection area such that:
1. The area is rendered in an intuitive way
2. All handles within the area when the click is released become selected, regardless if they were previously selected or not.
8. Click+dragging an unselected handle selects it and then drags it along with the mouse.
9. Shift+clicking+dragging any handle selects and then drags all selected handles along with the mouse.

There's also another case for when there are multiple potentially-overlapping handles: Clicking in one location multiple times cycles through the handles that could be selected. Additionally, all selection and drag operations need to be undoable and redoable.

You can see that dissecting what seems like common, every-day mouse gestures yields a complex set of states and conditions that need to be dealt with. (I'm actually including the bulk of this driver code down below.) This functionality needs to be at least 99% defect-free, as it's essentially part of the kernel of any mouse-centric application. This makes it a great candidate for unit tests, and luckily mouse input is just data and there are tools for writing these unit tests (also included below).

The initial version of the selection algorithm was near-sighted and tangled in with the color-changing logic. It instantly fell apart when trying to implement the dragging functionality. It was necessary though, as it helped me map out the scenarios I listed above.

The second version is ultimately what I'm sticking with, though there are things about it I don't like. Both versions took me about a week to implement, and then the unit tests took me another week and a half or so. This isn't fun work either, and I feel like I have a type of PTSD from all of the system refactoring needed to get everything running smoothly. That being said, once all refactoring was done, the unit tests held up and so did the functionality. I started with vertex selection handles, and minus some minor selection bookkeeping adjustments, other types of selection handles fell right into place (triangle and edge).

All of this has left me exhausted. Exhausted and feeling a little hopeless. Is what I'm doing dumb or at all worth it? I think it will be in the end, but I think there's a lesson to be learned about balancing the "fun" part of a technical hobby with the "legitimate work" part; both parts are necessary, and neglecting one for too long for the other's sake is bound to intensify the eventual fatigue.

What's next? There are a lot of immediate next steps I could take:

1. Grid
2. Zooming
3. Improving the save system and "catching up" the functionality that isn't currently persisted/restored
4. Begin with brushes
5. Texture support

Addressing technical debt is always there. I'm one person, so the code is far from perfect--even for how I want it to be. I have to choose something with more bang for the buck while I refresh though. I'd like to choose either texture support or brush functionality, but zooming is also more of a "kernelly" thing that I feel needs to be established before I implement more advanced functionality. However, while looking into zooming, I realized a grid would be helpful for debugging the zoom (math)--and thus I'm going with the grid. The grid will be good, since it's forcing me to address other areas of technical debt in healthier, less grindy ways.

SelectionManager.ts
import { Layer } from "../engine/world/Layer";
import { SelectionModel, SelectionHandle, TertiarySelectionCategory, PrimarySelectionCategory, SecondarySelectionCategory } from "../engine/model/select/Selectable";
import { Engine } from "../engine/Engine";
import { World } from "../engine/world/World";
import { vec4, vec2 } from "gl-matrix";
import { Model } from "../engine/model/Model";
import { SelectionUtilities } from "../engine/model/select/SelectionUtilities";
import { DEFAULT_SHADER_NAME } from "../TestScene";
import { Pair } from "../../util/Pair";
import { Camera } from "../engine/render/Camera";
import { BoundingShape } from "../engine/core/BoundingShape";
import { Mesh } from "../engine/model/Mesh";
import { WireframeRenderMode, ShadingRenderMode } from "../engine/render/RenderMode";
import { Vertex } from "../engine/model/Vertex";
import { IDMap } from "../../util/IDMap";
import { ID } from "../engine/core/ID";
export const SELECTION_HANDLE_LAYER_ORDER: number = 10000;
export interface ClickState
{
startMousePosition: vec2;
wasDragging: boolean;
}
export class SelectionResult
{
public readonly handlesSelected: SelectionHandle[] = [];
public readonly handlesDeselected: SelectionHandle[] = [];
public readonly handlesMoved: SelectionHandle[] = [];
public readonly netAmountMoved: vec2 = vec2.create();
public readonly dragInitialMouseDownPosition: vec2 = vec2.create();
}
export class SelectionManager
{
private readonly selectionHandleLayer: Layer = new Layer();
//private readonly layersForEditingByName: Map<string, Layer> = new Map();
private readonly selectionModel: SelectionModel = new SelectionModel();
private readonly world: World;
private selectionHandlesAndModels: Pair<SelectionHandle, Model>[] = [];
private selectedHandlePairIndices: Set<number> = new Set();
private clickAndDragState: ClickState = null;
private handleIndices: number[];
private indexCycleIndex: number;
private readonly selectionResult: SelectionResult = new SelectionResult();
private handleSize: number;
private normalHandleColor: vec4;
private selectedHandleColor: vec4;
private readonly selectingArea: Pair<boolean, vec2> = { first: false, second: [ 0, 0 ] };
private readonly areaBoundingShape: BoundingShape = new BoundingShape();
private readonly areaModel: Model = new Model(
DEFAULT_SHADER_NAME,
new Mesh(),
{
color: /* TODO */ [ 0.3, 0.7, 0.3, 0.3 ],
paralaxFactor: 1.0,
visible: false,
wireframeRenderMode: WireframeRenderMode.NEVER,
shadingRenderMode: ShadingRenderMode.FLAT_FULL_BRIGHT
});
public constructor(
private readonly engine: Engine,
private readonly selectionHandleShaderName: string)
{
this.world = this.engine.getWorld();
const firstVertex: Vertex = this.areaModel.mesh.addVertex([ 0, 0 ]);
const secondVertex: Vertex = this.areaModel.mesh.addVertex([ 0, 0 ]);
const thirdVertex: Vertex = this.areaModel.mesh.addVertex([ 0, 0 ]);
const fourthVertex: Vertex = this.areaModel.mesh.addVertex([ 0, 0 ]);
this.areaModel.mesh.addTriangle(firstVertex.getID(), secondVertex.getID(), thirdVertex.getID());
this.areaModel.mesh.addTriangle(thirdVertex.getID(), fourthVertex.getID(), firstVertex.getID());
}
public getSelectionHandleLayer(): Layer
{
return this.selectionHandleLayer;
}
public setHandleSize(handleSize: number): void
{
this.handleSize = handleSize;
// TODO: Just update handle models, not rebuild the entire thing. Oh well this.updateHandles(false);
}
public setNormalHandleColor(normalHandleColor: vec4): void
{
this.normalHandleColor = normalHandleColor;
this.updateHandles(false);
}
public setSelectedHandleColor(selectedHandleColor: vec4): void
{
this.selectedHandleColor = selectedHandleColor;
this.updateHandles(false);
}
public setPrimaryCategory(primaryCategory: PrimarySelectionCategory): void
{
this.selectionModel.setPrimaryCategory(primaryCategory);
this.updateHandles(true);
}
public setSecondaryCategory(secondaryCategory: SecondarySelectionCategory): void
{
this.selectionModel.setSecondaryCategory(secondaryCategory);
this.updateHandles(true);
}
public setTertiaryCategory(tertiaryCategory: TertiarySelectionCategory): void
{
this.selectionModel.setTertiaryCategory(tertiaryCategory);
this.updateHandles(true);
}
/*public addLayerForEditing(layer: Layer): void { this.layersForEditingByName.set(layer.getName(), layer);
this.updateHandles(); }*/
/*public removeLayerForEditing(layer: Layer): void { const layerName: string = layer.getName(); if (this.layersForEditingByName.has(layerName)) { this.layersForEditingByName.delete(layerName);
this.updateHandles(); } }*/
public handleSelectionAndDragging(
currentPosition: vec2,
primaryButtonDown: boolean,
shiftKeyPressed: boolean,
engineDebug: boolean): SelectionResult
{
this.selectionResult.handlesSelected.splice(0, this.selectionResult.handlesSelected.length);
this.selectionResult.handlesDeselected.splice(0, this.selectionResult.handlesDeselected.length);
this.selectionResult.handlesMoved.splice(0, this.selectionResult.handlesMoved.length);
if (this.clickAndDragState != null)
{
this.handleUpdateInClickOrDrag(currentPosition, primaryButtonDown, shiftKeyPressed, engineDebug);
}
else if (this.selectingArea.first)
{
this.handleSelectingArea(currentPosition, primaryButtonDown, shiftKeyPressed, engineDebug);
}
else {
this.handleNoState(currentPosition, primaryButtonDown, shiftKeyPressed, engineDebug);
}
this.updateHandles(false);
this.updateHandleColors();
return this.selectionResult;
}
private handleUpdateInClickOrDrag(
currentPosition: vec2,
primaryButtonDown: boolean,
shiftKeyPressed: boolean,
engineDebug: boolean): void
{
if (engineDebug)console.log(`clickAndDragState not null`);
// How far did the mouse move? const mouseDelta: vec2 = vec2.subtract(vec2.create(), currentPosition, this.clickAndDragState.startMousePosition);
const pixelsMoved: number = (Math.abs(mouseDelta[0]) + Math.abs(mouseDelta[1]));
// Not too far, it's part of a click or the end of dragging. if (pixelsMoved == 0)
{
this.handleMouseDidNotMoveInClickOrDrag(currentPosition, primaryButtonDown, shiftKeyPressed, engineDebug);
}
// Too far, it's the start of dragging. else {
this.handleMouseMovedInClickOrDrag(currentPosition, primaryButtonDown, shiftKeyPressed, mouseDelta, pixelsMoved, engineDebug);
}
}
private handleSelectingArea(
currentPosition: vec2,
primaryButtonDown: boolean,
shiftKeyPressed: boolean,
engineDebug: boolean): void
{
if (engineDebug)console.log(`selectingArea`);
const selectedHandles: SelectionHandle[] = this.handleAreaSelection(currentPosition, primaryButtonDown, shiftKeyPressed);
this.selectionResult.handlesSelected.push(...selectedHandles);
}
private handleNoState(
currentPosition: vec2,
primaryButtonDown: boolean,
shiftKeyPressed: boolean,
engineDebug: boolean): void
{
const handleIndices: number[] = this.findSelectionHandleIndices(currentPosition, primaryButtonDown);
if (engineDebug)console.log(`no other state`);
if (primaryButtonDown)
{
if (engineDebug)console.log(` primary button down`);
if (handleIndices.length > 0)
{
if (engineDebug)console.log(` handleIndices under mouse ${handleIndices.length}`);
this.clickAndDragState = {
startMousePosition: vec2.clone(currentPosition),
wasDragging: false };
this.handleIndices = handleIndices;
this.indexCycleIndex = 0;
vec2.copy(this.selectionResult.dragInitialMouseDownPosition, currentPosition);
}
else {
if (engineDebug)console.log(` handleIndices is 0`);
// No handles under the mouse, but the Shift key being down triggers selecting an area if (shiftKeyPressed)
{
if (engineDebug)console.log(` shift key pressed`);
this.selectingArea.first = true;
vec2.copy(this.selectingArea.second, currentPosition);
this.updateAreaSelectionModel(this.selectingArea.second);
this.areaModel.renderProperties.visible = true;
}
else {
if (engineDebug)console.log(` shift key not pressed`);
const deselectedHandles: SelectionHandle[] = this.clearSelection(true);
this.selectionResult.handlesDeselected.push(...deselectedHandles);
}
}
}
}
private updateHandles(handlesChanged: boolean): void
{
this.selectionHandleLayer.clearModels();
if (handlesChanged)
{
for (let ii = 0; ii < this.selectionHandlesAndModels.length; ii++)
{
const handle: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[ii];
handle.first.setSelected(false);
handle.first.setHighlighted(false);
}
this.selectedHandlePairIndices.clear();
this.clickAndDragState = null;
this.handleIndices = null;
this.indexCycleIndex = 0;
this.selectionResult.handlesSelected.splice(0, this.selectionResult.handlesSelected.length);
this.selectionResult.handlesDeselected.splice(0, this.selectionResult.handlesDeselected.length);
this.selectionResult.handlesMoved.splice(0, this.selectionResult.handlesMoved.length);
vec2.zero(this.selectionResult.netAmountMoved);
}
//if (handles.length > 0) const handles: SelectionHandle[] = this.world.getSelectionHandles(this.selectionModel);
// TODO: Switching the set of handles causes issues with the selection pipeline. // All of the bookkeeping gets out of sync. // This is why the SelectionManager should not maintain the list of handles at all. this.selectionHandlesAndModels = SelectionUtilities.createModelFromHandles(
handles,
DEFAULT_SHADER_NAME,
this.handleSize,
this.normalHandleColor,
this.selectedHandleColor);
for (let ii = 0; ii < this.selectionHandlesAndModels.length; ii++)
{
const selectionHandleAndModel: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[ii];
this.selectionHandleLayer.addModel(selectionHandleAndModel.second);
}
this.selectionHandleLayer.addModel(this.areaModel);
}
private handleMouseDidNotMoveInClickOrDrag(
currentPosition: vec2,
primaryButtonDown: boolean,
shiftKeyPressed: boolean,
engineDebug: boolean): void
{
if (engineDebug)console.log(` pixelsMoved == 0`);
// Mouse button was and is still down? Report back about the handles that were moved. if (primaryButtonDown)
{
if (engineDebug)console.log(` primary button down`);
}
// Mouse button no longer down, this is a click else if (!this.clickAndDragState.wasDragging)
{
if (engineDebug)console.log(` primary button not down and was not just dragging`);
const handleIndex: number = this.handleIndices[this.indexCycleIndex];
if (shiftKeyPressed)
{
if (engineDebug)console.log(` shift key pressed (toggling selection on ${handleIndex} and ending mouse click event)`);
if (this.selectionHandlesAndModels[handleIndex].first.getIsSelected())
{
this.removeSelection(handleIndex);
this.selectionResult.handlesDeselected.push(this.selectionHandlesAndModels[handleIndex].first);
}
else {
this.addSelection(handleIndex);
this.selectionResult.handlesSelected.push(this.selectionHandlesAndModels[handleIndex].first);
}
this.clickAndDragState = null;
}
else {
if (engineDebug)console.log(` shift key not pressed (clearing selection, then selecting ${handleIndex})`);
const deselectedHandles: SelectionHandle[] = this.clearSelection(false);
const handle: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[handleIndex];
this.addSelection(handleIndex);
this.selectionResult.handlesDeselected.push(...deselectedHandles);
this.selectionResult.handlesSelected.push(handle.first);
}
// Cycle to the handle, in case the user is clicking in the same spot (to cycle the handle) this.indexCycleIndex = ((this.indexCycleIndex + 1) % this.handleIndices.length);
}
else {
this.addMovedHandlesToResult(currentPosition);
this.clickAndDragState = null;
}
}
private handleMouseMovedInClickOrDrag(
currentPosition: vec2,
primaryButtonDown: boolean,
shiftKeyPressed: boolean,
mouseDelta: vec2,
pixelsMoved: number,
engineDebug: boolean): void
{
if (engineDebug)console.log(` pixelsMoved > 0 (${pixelsMoved})`);
// Mouse button still down: This is a drag
if (primaryButtonDown)
{
if (engineDebug)console.log(` primary button down`);
if (!this.clickAndDragState.wasDragging)
{
// We know this is the start of dragging because wasDragging is false vec2.copy(this.selectionResult.dragInitialMouseDownPosition, this.clickAndDragState.startMousePosition);
}
// This ensures that the "mouse button up" event doesn't single-select the vertex that was used to drag. this.clickAndDragState.wasDragging = true;
// It's possible for nothing to be selected yet, so grab the first handle index and select it. // The handle index is guaranteed to be there because we wouldn't be in the "clickAndDragState" if there weren't handles under the mouse when setting it to not null. const handleIndex: number = this.handleIndices[0];
const handle: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[handleIndex];
if (!handle.first.getIsSelected())
{
if (!shiftKeyPressed)
{
const deselectedHandles: SelectionHandle[] = this.clearSelection(false);
this.selectionResult.handlesDeselected.push(...deselectedHandles);
}
this.addSelection(handleIndex);
this.selectionResult.handlesSelected.push(handle.first);
}
this.selectedHandlePairIndices.forEach(index => {
const handle: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[index];
handle.first.translated(mouseDelta);
});
this.clickAndDragState.startMousePosition = currentPosition;
}
// Mouse button not down anymore else {
if (engineDebug)console.log(` primary button not down`);
// This event is over. this.clickAndDragState = null;
// Report back about the handles that were moved. this.addMovedHandlesToResult(currentPosition);
}
}
private findSelectionHandleIndices(currentPosition: vec2, primaryButtonDown: boolean): number[]
{
let indices: number[] = [];
for (let ii = 0; ii < this.selectionHandlesAndModels.length; ii++)
{
const selectionHandleAndModel: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[ii];
if (selectionHandleAndModel.first.boundingShape.pointIsIn(currentPosition))
{
selectionHandleAndModel.first.setHighlighted(true);
indices.push(ii);
}
else {
selectionHandleAndModel.first.setHighlighted(false);
}
}
return indices;
}
private addMovedHandlesToResult(currentPosition: vec2): void
{
vec2.subtract(this.selectionResult.netAmountMoved, currentPosition, this.selectionResult.dragInitialMouseDownPosition);
if (this.clickAndDragState &&
((Math.abs(this.selectionResult.netAmountMoved[0]) > 0) || (Math.abs(this.selectionResult.netAmountMoved[1]) > 0)))
{
this.selectedHandlePairIndices.forEach(index => {
const handle: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[index];
this.selectionResult.handlesMoved.push(handle.first);
});
}
}
private handleAreaSelection(currentPosition: vec2, primaryButtonDown: boolean, shiftKeyPressed: boolean): SelectionHandle[]
{
this.updateAreaSelectionModel(currentPosition);
const selectedHandles: SelectionHandle[] = [];
if (primaryButtonDown && shiftKeyPressed)
{
for (let ii = 0; ii < this.selectionHandlesAndModels.length; ii++)
{
const selectionHandleAndModel: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[ii];
if (selectionHandleAndModel.first.boundingShape.boxesCollide(this.areaBoundingShape))
{
selectionHandleAndModel.second.renderProperties.color = this.selectedHandleColor;
selectionHandleAndModel.first.setHighlighted(true);
}
else {
if (!selectionHandleAndModel.first.getIsSelected())
{
selectionHandleAndModel.second.renderProperties.color = this.normalHandleColor;
}
selectionHandleAndModel.first.setHighlighted(false);
}
}
}
else {
this.selectingArea.first = false;
this.areaModel.renderProperties.visible = false;
for (let ii = 0; ii < this.selectionHandlesAndModels.length; ii++)
{
const selectionHandleAndModel: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[ii];
selectionHandleAndModel.first.setHighlighted(false);
if (!selectionHandleAndModel.first.getIsSelected())
{
selectionHandleAndModel.second.renderProperties.color = this.normalHandleColor;
}
if (selectionHandleAndModel.first.boundingShape.boxesCollide(this.areaBoundingShape))
{
this.addSelection(ii);
selectedHandles.push(selectionHandleAndModel.first);
}
}
}
return selectedHandles;
}
private updateAreaSelectionModel(currentPosition: vec2): void
{
this.areaBoundingShape.calculateFrom([ this.selectingArea.second, currentPosition ]);
const areaBoundingShapeLowerLeft: vec2 = this.areaBoundingShape.getBoxLowerLeft();
const areaBoundingShapeUpperRight: vec2 = this.areaBoundingShape.getBoxUpperRight();
const _areaModelVertexPositions: IDMap<vec2> = this.areaModel.mesh.getVertexPositions();
const areaModelVertexPositions: vec2[] = _areaModelVertexPositions.valuesAsArray();
const firstVertexPosition: vec2 = areaModelVertexPositions[0];
const secondVertexPosition: vec2 = areaModelVertexPositions[1];
const thirdVertexPosition: vec2 = areaModelVertexPositions[2];
const fourthVertexPosition: vec2 = areaModelVertexPositions[3];
firstVertexPosition[0] = areaBoundingShapeLowerLeft[0];
firstVertexPosition[1] = areaBoundingShapeLowerLeft[1];
secondVertexPosition[0] = areaBoundingShapeLowerLeft[0];
secondVertexPosition[1] = areaBoundingShapeUpperRight[1];
thirdVertexPosition[0] = areaBoundingShapeUpperRight[0];
thirdVertexPosition[1] = areaBoundingShapeUpperRight[1];
fourthVertexPosition[0] = areaBoundingShapeUpperRight[0];
fourthVertexPosition[1] = areaBoundingShapeLowerLeft[1];
}
private clearSelection(clearHighlightedHandles): SelectionHandle[]
{
const deselectedHandles: SelectionHandle[] = [];
this.selectedHandlePairIndices.forEach(index => {
const selectionHandleAndModel: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[index];
if (selectionHandleAndModel.first.getIsSelected())
{
deselectedHandles.push(selectionHandleAndModel.first);
}
selectionHandleAndModel.second.renderProperties.color = this.normalHandleColor;
selectionHandleAndModel.first.setSelected(false);
if (clearHighlightedHandles)
{
selectionHandleAndModel.first.setHighlighted(false);
}
});
this.selectedHandlePairIndices.clear();
return deselectedHandles;
}
public select(handle: SelectionHandle): boolean
{
const handleIndex: number = this.findIndexOfSelectionHandle(handle);
if (handleIndex != -1)
{
this.addSelection(handleIndex);
return true;
}
return false;
}
public deselect(handle: SelectionHandle): boolean
{
const handleIndex: number = this.findIndexOfSelectionHandle(handle);
if (handleIndex != -1)
{
this.removeSelection(handleIndex);
return true;
}
return false;
}
private findIndexOfSelectionHandle(handle: SelectionHandle): number
{
for (let ii = 0; ii < this.selectionHandlesAndModels.length; ii++)
{
const handleAndModel: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[ii];
if (handleAndModel.first == handle)
{
return ii;
}
}
return -1;
}
private addSelection(handlePairIndex: number): void
{
this.selectedHandlePairIndices.add(handlePairIndex);
this.selectionHandlesAndModels[handlePairIndex].second.renderProperties.color = this.selectedHandleColor;
this.selectionHandlesAndModels[handlePairIndex].first.setSelected(true);
}
private removeSelection(handlePairIndex: number): void
{
this.selectedHandlePairIndices.delete(handlePairIndex);
this.selectionHandlesAndModels[handlePairIndex].second.renderProperties.color = this.normalHandleColor;
this.selectionHandlesAndModels[handlePairIndex].first.setSelected(false);
}
private updateHandleColors(): void
{
for (let ii = 0; ii < this.selectionHandlesAndModels.length; ii++)
{
const handle: Pair<SelectionHandle, Model> = this.selectionHandlesAndModels[ii];
if (handle.first.shouldHighlight())
{
handle.second.renderProperties.color = this.selectedHandleColor;
}
else {
handle.second.renderProperties.color = this.normalHandleColor;
}
}
}
}

World Structure

Warning: This post is long.

World Structure

Despite what I said previously about not posting about world structure, I decided to go ahead and do it anyways. That being said:

What is a world structure?

Do I mean the layout of the world; the data structures the engine uses to render the scene, run physics, know what sounds to play, and what entities to maintain; or something else? Well, it's kind of all of the above. It almost feels nebulous and hard to pin down, and stuff like resource management and "loading areas" (think the elevators in the Metroid games) add complexity to how it should all be organized. That is, at least until I start thinking about what I want the game I'm designing to actually be like. So, maybe that's the question I should be thinking about more.

I know I'm making an engine, but what game do I want to make?

Well, here are some things I know:

1. It's going to be a 2D Metroidvania with a very large, interconnected world and multiple possible paths through the main story.
2. Model transformations need to occur on the CPU (not the GPU) so other engine features can exploit them post-transformation.
3. It will have multiple independent background and foreground layers--all with parallax relative to the "main" layer.
4. Background and foreground layers will also have entities and physics to bring them to life.
5. All lights will have both ambient (non-physical) values and cast dynamic shadows--even between layers.
6. There will be other environmental and lighting effects, such as fog, precipitation, and high dynamic range.

Of course, there are probably thousands of things I could list here--all on varying levels of detail. My reasoning for mentioning the items on this list are their relationships to the world structure. Let's just work our way down this list.

It's going to be a 2D Metroidvania with a very large, interconnected world and multiple possible paths through the main story.

Right off the bat, I know the world will need to split into different larger areas, perhaps even with names (like "Brinstar" and "Norfair" in Metroid). The data structures should be conducive to a load area approach or seamless/dynamic loading approach; this means that they should at least keep track of entities within them for live engine bookkeeping. For lack of better words, let's call these larger areas zones.

What are the contents of a zone? Thinking back to Super Metroid again, I would say the main thing is "screens"--or the areas that doors separate. I think background and foreground layers are also an obvious component--especially when thinking about outdoor and large cavernous areas. I don't personally like "screen" as my noun though. It's kind of a misnomer about the implied behavior as some of them go beyond one literal "screen" (as in, the display/monitor)--sometimes multiple times so. To pay homage to my level design upbringing, I'll call these things sectors.

This is making sense so far. Zones are made of layers (background and foreground) and sectors. This means layers are "shared" across sectors. I think this adds up: for example, a background layer containing a mountain range should "parallax" across all sectors in the zone. Another background layer in front of the first one can act as depth-wise boundaries/windows occluding or revealing the mountain range. The mountain range background layer can have live weather, passive entities running through scripted sequences (such as birds flying around), trees that respond to random wind patterns, etc. The layer in front of the mountain range layer can occlude (hide) it in certain parts, simply by having content in them.

Model transformations need to occur on the CPU (not the GPU) so other engine features can exploit them post-transformation.

Probably the main motivator here is that I'm a functional newbie with OpenGL and shader programming. I'm sure it's possible to divert all of the post-model-transformation functionality to shaders, but that's thinking on a level I'm not at for Integral v1.0. Some examples of post-model-transformation functionalities:

1. Animations actually conform to world geometry. Think of your foot slanting down when walking down a hill and how that affects your gait and stride. I want the actors in Integral to have altered gait and stride in such circumstances as well.
2. Lighting is influenced by dynamic geometry beyond rendering shadows. Light and shadow volumes must both affect high dynamic range conditions and render properties of environmental effects.
3. Finalization effects such as fur and foliage need feedback from the actively-transformed models and physics system to know when to spawn further particle effects--such as fur flying when attacked.

What does this mean in practicality? It's simple actually: Models must preserve a default state and maintain an active state for their geometry. (I will talk about the model structures more in-depth in a later post.)

It will have multiple independent background and foreground layers--all with parallax relative to the "main" layer.

This one isn't too bad. All zones have an ordered list (or just a list) of background and foreground layers. Every layer has a set of understood render properties applied to every renderable item in them--including parallax.

Parallax, simply put, is the ratio of translation relative to the camera. For example, a parallax of 0.9 means that for every 10 units the camera moves to the right, the objects with that parallax appear to "move left" 9 units. The standard parallax is 1.0. Parallax values of less than 1.0 make objects appear further away, and parallax values of greater than 1.0 make objects appear closer.

Background and foreground layers will also have entities and physics to bring them to life.

An entity is anything that has game logic executed every frame. I borrowed this terminology from the Quake and Half-Life games. A synonym is actor.

So, background and foreground layers will have these as well as the "main" layer (which isn't actually technically a layer). This allows the layers to appear more dynamic ("come to life") as per the rules and hooks established by the engine.

Without special triggers, entities also won't be able to interact with each other across layers. This reduces special logic needed to keep scripted sequences running as intended--i.e. so entities from different layers don't accidentally interact with each other.

In addition to entities, layers will also have static world models (also known as brushes) for the scenery. I'm contemplating introducing the concept of detail brushes that don't execute special logic per frame yet still have world and player interactions. Applications of these would be foliage and other things that have limited, pseudo-scripted interactions such as "play sound abc.wav and run physics on certain sub-models when an entity touches it". My hesitation, at least right now, is that it could muddy the kernel of the engine and end up being frustrating when actually designing content.

All lights will have both ambient (non-physical) values and cast dynamic shadows--even between layers.

More importantly for the world structure: There will be high dynamic range (HDR) functionality, and lighting will influence it. How this will work example is actually a little unclear to me still, but I know that the engine will need to keep track of "dark" areas versus "bright" areas. I'm no light expert, but empirically I know that being outside on a bright day and then going inside--even to the best- or most naturally-lit of rooms--everything appears darker, tinted, and kind of "lower quality" while my eyes adjust. And on the flip side--going from inside to outside on a bright day--everything appears saturated until my eyes adjust.

Thinking about these scenarios and how to emulate them: sectors need rooms. Actually, every sector will have at least one (parent) room and arbitrarily-deep nested (children) rooms. Additionally though, rooms will need different models and render properties for when the player is inside or outside of them (or exterior and interior models). Think of this example: you're walking along a (side-scrolling, Metroidvania) street and approach a house. It should be possible to render that house differently when outside of it versus inside of it for what I feel are obvious creative reasons. The entirety of the house would be one (child) room, and the rooms inside the house would be further-nested rooms.

The exterior versus interior concept also has implications for the next point.

There will be other environmental and lighting effects, such as fog, precipitation, and high dynamic range.

In a sense, this point kind of ties together previous points. In short, the world structures at their core need to facilitate these features through game mechanic enrichment. The renderer needs to know when and where to render environmental effects like fog and precipitation; since it will know how to render zones, sectors, and rooms anyways, it can use these structures (primarily rooms in this case) to make these types of decisions. An example of this is a zone where it always rains, but that rain doesn't appear in or interact with the interior of rooms.

Conclusion

Having well-established world structures has really helped lay out how the renderer works. As a matter of fact, my Hello World Triangle (seen in last week's post) is built on a mesh used by a model that's placed in the background layer of a zone added to the world. The model references a previously-setup shader by name, as there's a global shader manager that makes it possible to have multiple renderers--all with different WebGL canvases (also more on this in a later post).

Angular, Starting Out, Weird Performance Issues

I've been a professional software developer/engineer/programmer/whatever-role-title for 11 1/2 years. The types of projects I've worked on professionally have weaved in and out of academia, defense, enterprise, web, and traditional/desktop applications (in all combinations). My attitude and relationship with web development has been pretty complicated historically-speaking, so much that I scrapped several drafts of this post because they became too ranty.

Long story short: I decided to go with Angular, TypeScript, and Electron as the core technologies used in Integral. Why target the browser for what's intended to eventually be a professional game engine? Well, basically every device has a WebGL-capable and controller-ready browser built in. There's no cross-compiling necessary, and WebGL standardizes the interfacing for testing and using extended OpenGL features. Electron wraps it all up as a "native" application, including native functionality like actual file system IO (via IPC--or interprocess communication--APIs). One of my goals for Integral all along has been to be highly cross-platform, going as far as editing content on my phone and being able to play my games on any console. (As a matter of fact, I did a little bit of development for my first browser-based attempt on my phone using Termux (a Linux emulator) and a BlueTooth keyboard.) It honestly just makes sense, and it helps that I really dove in to Angular for my day job in the past year.

Even though I'm just a one-person team, I've been trying to take an agile-esque approach by breaking work down into proper user story-style tickets. I've found that using an actual ticketing system helps with the mental processing here, versus past attempts using large bullet lists in a Google Doc (or the like). As projects of this scale go, there have also been plenty of hurdles at every step, and I've been making sure to refactor and address other technical debt often. These things slow implementing features, but establish and maintain a cohesiveness to the entire codebase.

Recently, I've been working my way passed the "Hello World Triangle" phase into actually interacting with the triangle. The abstraction I've come up for a unified editing/selection model is: Handles represent different objects (i.e. entire triangles, entire edges, individual vertices, entire entities, etc.), and different editing and selection modes yield different handles. That way, the input system can interact with handles in (here I say it again) a unified way.

I haven't posted about how the world is organized, how the renderer works, or any other "dry" details because they're not finalized yet. I think I have them solid for the time being, but not to the level that I'd feel comfortable blogging about them. So let's just start from: Hey, I finally have some (selection) handles for the three vertices of my one glorious triangle:

(To toot my own horn a little bit, this screenshot shows functional persistence, input/camera-move, texture management, and rendering systems. The wireframes are can be toggled off entirely or selected between exterior edges (silhouettes), interior edges, or both. The "shaded" geometry can also be toggled on and off, though right now everything is Flat Full-Bright or off.)

When I finally implemented my handle highlighting logic (complete with that, "Ah yes, this is it!" feeling) and switched windows to test it, I noticed that the handle wouldn't really change colors until the mouse stopped moving while hovering over it:

Long story short, this problem about killed me (or less dramatically, short-circuited the project). I added in log statements everywhere, measuring the start-to-finish milliseconds for the render methods, mouse event methods, mouse-in-handle collision detection, etc. I saw everything happening in perfect sequence: event fires at x millis, handle collision detection at x+1 millis, render at x+2 millis. I even added checks to the render methods to print special logs if the object's color matched the highlight handle color, and those occurred far before I would actually see the change. I was desperate, until I ran the Chrome (Electron) profiler and found zone-evergreen consuming up to 87% of the cpu cycles.

zone-evergreen... isn't that... Angular! Zone, as far as Angular and most people are concerned, is the thing that triggers change detections so the backing data models and HTML components can automagically stay in sync. Of course this thing would be a hog, and my research told me it was running every single mouse event!

I toyed around with different ways to optimize Zone for my purposes. Nothing really worked in those regards. I then tried just disabling Zone (and thus change detection) all together, and my first test afterwards was the biggest sigh of relief I've had for this project--it finally worked. My beautiful handles highlighted green with mouse hover, and then went back to purple again. But oops, change detection was now off, so I dug further by leaving it on and removing one layer of components at a time. As it turns out, it was actually something in the Texture Manager (on the left, where you can see the crude "Beautiful Chair" image)--I suspect because are also components in it that listen for mouse events.

What I ended up doing was removing the Texture Manager components from the change detection hierarchy and triggering it manually when specific events occur. It works like butter.

Games I Find Fun, Integral Engine, 2D+

Like most in my age range, I grew up playing the classics. The games that always stuck with me the most were the Metroid games though. I loved the 2D ones, the Prime ones, and I even thoroughly enjoyed Other M until we moved to Maryland and I lost track of it. (I did give Hunters a fair chance. Like, I really tried. I just couldn't though.) So naturally, a "Metroidvania" style game is where my heart and focus have been in all of my post-Half-Life attempts. There is something nice and resolved of the simplicity, and the genre is certainly a departure from anything I ever worked on for Half-Life.

Now, I don't know about you, but when I fall for a game I fall hard. For the past couple of years it's been Overwatch for me, specifically on the X-Box. (The Overwatch character Brigitte is actually my inspiration for this screen name.) The unfortunate thing about multiplayer games in general is, they are stressful. I'm a yeller when it comes to game frustration, and I've definitely had some of my ugliest moments ever on the voice chat. Anymore, I have to enjoy the game in very small doses and keep a stern assessment on my overall experience. Good game, but lost--how jittery do I feel, or did I yell at all? etc.

What I'm getting to is, I think the contemporary focus on multiplayer games is a multifaceted whammy, and I find it distracting from the art form of game development. I have the most fun playing single-player (or perhaps local co-op), story-driven games with relatively static content and low-grind factor. Like, a Metroidvania!

So local (co-op) 2D games are what I will keep in scope. Thus, my first major architectural decision: Crossing off 3D-everything. This early decision frees up further architectural decisions across all boards, but thinking through 2D game content creation cycles raises a lot of flags for me. In terms of my previous post--tools, workflows, etc. that I want--I refuse to accept the static-leaning nature of 2D content. I want my characters to be truly animated and to actually interact with the environment, not just endless arrays of static sprites. And thus, my second major architectural decision: all non-static content will be driven by textured meshes and skeletal animations, and animations will be informed (constrained) by the environment.

One thing that always bugged me when developing content was all of the different tools you'd have to use just to get your content in a game. One unaffordable model editor for "models" (which are treated differently than the world), an unaffordable plugin for it just for skeletal animations, another tool for world editing, tools for managing whatever proprietary archive files the game uses, custom texture formats, etc. It's a mess, at least the last time I was seriously doing it (which was admittedly a long time ago). For my games and my development cycles, I want something integrated: tools not only meant for my engine, but that are literally part of the game run-time. So, the engine will contain content creation tools that can be used to create and adjust content in the middle of running a game. And there we have the codename I've given the engine: Integral.