Skip to content

Adding framebuffer support for filter() + CreateFilterShader for 2D mode #6559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Nov 23, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 0 additions & 7 deletions lib/empty-example/sketch.js

This file was deleted.

22 changes: 22 additions & 0 deletions src/core/p5.Renderer2D.js
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,28 @@ class Renderer2D extends p5.Renderer{
this._pInst._setProperty('drawingContext', this.drawingContext);
}

getFilterGraphicsLayer() {
// create hidden webgl renderer if it doesn't exist
if (!this.filterGraphicsLayer) {
// the real _pInst is buried when this is a secondary p5.Graphics
const pInst =
this._pInst instanceof p5.Graphics ?
this._pInst._pInst :
this._pInst;

// create secondary layer
this.filterGraphicsLayer =
new p5.Graphics(
this.width,
this.height,
constants.WEBGL,
pInst
);
}

return this.filterGraphicsLayer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this, we might want to do a similar size matching to what we do in RendererGL with its framebuffer so that we ensure this.filterGraphicsLayer matches width, height, and pixel density to this._pInst. Right now filters on 2D mode don't seem to work when resizing: https://editor.p5js.org/davepagurek/sketches/OOKkLOINp

Maybe we can also copy and paste the current unit test that checks that the framebuffer gets resized in WebGL mode, and adapt it to check the size of filterGraphicsLayer in 2D mode? It would definitely be helpful to leave behind as many tests as we can so that we don't have to rely so much on thinking of new things to test the next time we make changes to this system.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, i will do the changes asap.

}

_applyDefaults() {
this._cachedFillStyle = this._cachedStrokeStyle = undefined;
this._cachedBlendMode = constants.BLEND;
Expand Down
42 changes: 17 additions & 25 deletions src/image/pixels.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@
import p5 from '../core/main';
import Filters from './filters';
import '../color/p5.Color';
import * as constants from '../core/constants';

/**
* An array containing the color of each pixel on the canvas. Colors are
Expand Down Expand Up @@ -544,6 +543,15 @@ p5.prototype._copyHelper = (
* </div>
*/

/**
* @method getFilterGraphicsLayer
* @private
* @returns {p5.Graphics}
*/
p5.prototype.getFilterGraphicsLayer = function() {
return this._renderer.getFilterGraphicsLayer();
};

/**
* @method filter
* @param {Constant} filterType
Expand All @@ -560,7 +568,7 @@ p5.prototype.filter = function(...args) {
let { shader, operation, value, useWebGL } = parseFilterArgs(...args);

// when passed a shader, use it directly
if (shader) {
if (this._renderer.isP3D && shader) {
p5.RendererGL.prototype.filter.call(this._renderer, shader);
return;
}
Expand All @@ -586,27 +594,11 @@ p5.prototype.filter = function(...args) {

// when this is P2D renderer, create/use hidden webgl renderer
else {
// create hidden webgl renderer if it doesn't exist
if (!this.filterGraphicsLayer) {
// the real _pInst is buried when this is a secondary p5.Graphics
const pInst =
this._renderer._pInst instanceof p5.Graphics ?
this._renderer._pInst._pInst :
this._renderer._pInst;

// create secondary layer
this.filterGraphicsLayer =
new p5.Graphics(
this.width,
this.height,
constants.WEBGL,
pInst
);
}
const filterGraphicsLayer = this.getFilterGraphicsLayer();

// copy p2d canvas contents to secondary webgl renderer
// dest
this.filterGraphicsLayer.copy(
filterGraphicsLayer.copy(
// src
this._renderer,
// src coods
Expand All @@ -616,14 +608,14 @@ p5.prototype.filter = function(...args) {
);
//clearing the main canvas
this._renderer.clear();
// Resetting the matrix of the canvas

this._renderer.resetMatrix();
// filter it with shaders
this.filterGraphicsLayer.filter(operation, value);
filterGraphicsLayer.filter(...args);

// copy secondary webgl renderer back to original p2d canvas
this._renderer._pInst.image(this.filterGraphicsLayer, 0, 0);
this.filterGraphicsLayer.clear(); // prevent feedback effects on p2d canvas
this._renderer._pInst.image(filterGraphicsLayer, 0, 0);
filterGraphicsLayer.clear(); // prevent feedback effects on p2d canvas
}
};

Expand Down Expand Up @@ -927,4 +919,4 @@ p5.prototype.updatePixels = function(x, y, w, h) {
this._renderer.updatePixels(x, y, w, h);
};

export default p5;
export default p5;
54 changes: 22 additions & 32 deletions src/webgl/material.js
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ import './p5.Texture';
* @alt
* zooming Mandelbrot set. a colorful, infinitely detailed fractal.
*/
p5.prototype.loadShader = function(
p5.prototype.loadShader = function (
vertFilename,
fragFilename,
callback,
Expand Down Expand Up @@ -194,8 +194,7 @@ p5.prototype.loadShader = function(
* @alt
* zooming Mandelbrot set. a colorful, infinitely detailed fractal.
*/
p5.prototype.createShader = function(vertSrc, fragSrc) {
this._assert3d('createShader');
p5.prototype.createShader = function (vertSrc, fragSrc) {
p5._validateParameters('createShader', arguments);
return new p5.Shader(this._renderer, vertSrc, fragSrc);
};
Expand All @@ -204,13 +203,13 @@ p5.prototype.createShader = function(vertSrc, fragSrc) {
* Creates a new <a href="#/p5.Shader">p5.Shader</a> using only a fragment shader, as a convenience method for creating image effects.
* It's like <a href="#/createShader">createShader()</a> but with a default vertex shader included.
*
* <a href="#/createFilterShader">createFilterShader()</a> is intended to be used along with <a href="#/filter">filter()</a> for filtering the contents of a canvas in WebGL mode.
* <a href="#/createFilterShader">createFilterShader()</a> is intended to be used along with <a href="#/filter">filter()</a> for filtering the contents of a canvas.
* A filter shader will not be applied to any geometries.
*
* The fragment shader receives some uniforms:
* - `sampler2D tex0`, which contains the canvas contents as a texture
* - `vec2 canvasSize`, which is the width and height of the canvas
* - `vec2 texelSize`, which is the size of a pixel (`1.0/width`, `1.0/height`)
* - `vec2 canvasSize`, which is the p5 width and height of the canvas (not including pixel density)
* - `vec2 texelSize`, which is the size of a physical pixel including pixel density (`1.0/(width*density)`, `1.0/(height*density)`)
*
* For more info about filters and shaders, see Adam Ferriss' <a href="https://github.com/aferriss/p5jsShaderExamples">repo of shader examples</a>
* or the <a href="https://p5js.org/learn/getting-started-in-webgl-shaders.html">introduction to shaders</a> page.
Expand Down Expand Up @@ -277,14 +276,7 @@ p5.prototype.createShader = function(vertSrc, fragSrc) {
* </code>
* </div>
*/
p5.prototype.createFilterShader = function(fragSrc) {
let iswebgl;
if(this._renderer.GL){
iswebgl = true;
}else{
iswebgl = false;
}
this._assert3d('createFilterShader');
p5.prototype.createFilterShader = function (fragSrc) {
p5._validateParameters('createFilterShader', arguments);
let defaultVertV1 = `
uniform mat4 uModelViewMatrix;
Expand Down Expand Up @@ -328,13 +320,11 @@ p5.prototype.createFilterShader = function(fragSrc) {
`;
let vertSrc = fragSrc.includes('#version 300 es') ? defaultVertV2 : defaultVertV1;
const shader = new p5.Shader(this._renderer, vertSrc, fragSrc);
let target;
if(!iswebgl){
target = this._renderer.getFilterLayer();
}else{
target = this;
if (this._renderer.GL) {
shader.ensureCompiledOnContext(this);
} else {
shader.ensureCompiledOnContext(this._renderer.getFilterGraphicsLayer());
}
shader.ensureCompiledOnContext(target);
return shader;
};

Expand Down Expand Up @@ -423,7 +413,7 @@ p5.prototype.createFilterShader = function(fragSrc) {
* @alt
* canvas toggles between a circular gradient of orange and blue vertically. and a circular gradient of red and green moving horizontally when mouse is clicked/pressed.
*/
p5.prototype.shader = function(s) {
p5.prototype.shader = function (s) {
this._assert3d('shader');
p5._validateParameters('shader', arguments);

Expand Down Expand Up @@ -523,7 +513,7 @@ p5.prototype.shader = function(s) {
* Two rotating cubes. The left one is painted using a custom (user-defined) shader,
* while the right one is painted using the default fill shader.
*/
p5.prototype.resetShader = function() {
p5.prototype.resetShader = function () {
this._renderer.userFillShader = this._renderer.userStrokeShader = null;
return this;
};
Expand Down Expand Up @@ -658,7 +648,7 @@ p5.prototype.resetShader = function() {
* @alt
* quad with a texture, mapped using normalized coordinates
*/
p5.prototype.texture = function(tex) {
p5.prototype.texture = function (tex) {
this._assert3d('texture');
p5._validateParameters('texture', arguments);
if (tex.gifProperties) {
Expand Down Expand Up @@ -741,7 +731,7 @@ p5.prototype.texture = function(tex) {
* @alt
* quad with a texture, mapped using image coordinates
*/
p5.prototype.textureMode = function(mode) {
p5.prototype.textureMode = function (mode) {
if (mode !== constants.IMAGE && mode !== constants.NORMAL) {
console.warn(
`You tried to set ${mode} textureMode only supports IMAGE & NORMAL `
Expand Down Expand Up @@ -814,7 +804,7 @@ p5.prototype.textureMode = function(mode) {
* @alt
* an image of the rocky mountains repeated in mirrored tiles
*/
p5.prototype.textureWrap = function(wrapX, wrapY = wrapX) {
p5.prototype.textureWrap = function (wrapX, wrapY = wrapX) {
this._renderer.textureWrapX = wrapX;
this._renderer.textureWrapY = wrapY;

Expand Down Expand Up @@ -855,7 +845,7 @@ p5.prototype.textureWrap = function(wrapX, wrapY = wrapX) {
* @alt
* Sphere with normal material
*/
p5.prototype.normalMaterial = function(...args) {
p5.prototype.normalMaterial = function (...args) {
this._assert3d('normalMaterial');
p5._validateParameters('normalMaterial', args);
this._renderer.drawMode = constants.FILL;
Expand Down Expand Up @@ -968,7 +958,7 @@ p5.prototype.normalMaterial = function(...args) {
* as an array, or as a CSS string
* @chainable
*/
p5.prototype.ambientMaterial = function(v1, v2, v3) {
p5.prototype.ambientMaterial = function (v1, v2, v3) {
this._assert3d('ambientMaterial');
p5._validateParameters('ambientMaterial', arguments);

Expand Down Expand Up @@ -1039,7 +1029,7 @@ p5.prototype.ambientMaterial = function(v1, v2, v3) {
* as an array, or as a CSS string
* @chainable
*/
p5.prototype.emissiveMaterial = function(v1, v2, v3, a) {
p5.prototype.emissiveMaterial = function (v1, v2, v3, a) {
this._assert3d('emissiveMaterial');
p5._validateParameters('emissiveMaterial', arguments);

Expand Down Expand Up @@ -1125,7 +1115,7 @@ p5.prototype.emissiveMaterial = function(v1, v2, v3, a) {
* as an array, or as a CSS string
* @chainable
*/
p5.prototype.specularMaterial = function(v1, v2, v3, alpha) {
p5.prototype.specularMaterial = function (v1, v2, v3, alpha) {
this._assert3d('specularMaterial');
p5._validateParameters('specularMaterial', arguments);

Expand Down Expand Up @@ -1174,7 +1164,7 @@ p5.prototype.specularMaterial = function(v1, v2, v3, alpha) {
* @alt
* two spheres, one more shiny than the other
*/
p5.prototype.shininess = function(shine) {
p5.prototype.shininess = function (shine) {
this._assert3d('shininess');
p5._validateParameters('shininess', arguments);

Expand All @@ -1192,7 +1182,7 @@ p5.prototype.shininess = function(shine) {
* @param {Number[]} color [description]
* @return {Number[]]} Normalized numbers array
*/
p5.RendererGL.prototype._applyColorBlend = function(colors) {
p5.RendererGL.prototype._applyColorBlend = function (colors) {
const gl = this.GL;

const isTexture = this.drawMode === constants.TEXTURE;
Expand Down Expand Up @@ -1227,7 +1217,7 @@ p5.RendererGL.prototype._applyColorBlend = function(colors) {
* @param {Number[]} color [description]
* @return {Number[]]} Normalized numbers array
*/
p5.RendererGL.prototype._applyBlendMode = function() {
p5.RendererGL.prototype._applyBlendMode = function () {
if (this._cachedBlendMode === this.curBlendMode) {
return;
}
Expand Down
34 changes: 18 additions & 16 deletions src/webgl/p5.RendererGL.js
Original file line number Diff line number Diff line change
Expand Up @@ -1008,6 +1008,18 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
}
return this.filterGraphicsLayerTemp;
}
matchSize(fboToMatch, target) {
if (
fboToMatch.width !== target.width ||
fboToMatch.height !== target.height
) {
fboToMatch.resizeCanvas(target.width, target.height);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this needs to just be resize instead of resizeCanvas, this currently throws errors: https://editor.p5js.org/davepagurek/sketches/jOYUVNMXB

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no errors now

No errors now

}

if (fboToMatch.pixelDensity() !== target._pInst.pixelDensity()) {
fboToMatch.pixelDensity(target._pInst.pixelDensity());
}
}
filter(...args) {

let fbo = this.getFilterLayer();
Expand Down Expand Up @@ -1048,21 +1060,9 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
const target = this.activeFramebuffer() || this;

// Resize the framebuffer 'fbo' and adjust its pixel density if it doesn't match the target.
this.matchSize(fbo, target);

if (
fbo.width !== this.width ||
fbo.height !== this.height
) {
// Resize fbo
fbo.resizeCanvas(this.width, this.height);
}
if (
fbo.pixelDensity() !== this._pInst.pixelDensity()
) {
fbo.pixelDensity(this._pInst.pixelDensity());
}

// Set the yScale of the filterCamera for framebuffers.
// Set filterCamera for framebuffers.
if (target !== this) {
this.filterCamera = this.getFilterLayer().createCamera();
}
Expand All @@ -1073,10 +1073,12 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
1 / (target.height * target.pixelDensity())
];

// apply blur shader with multiple passes
// apply blur shader with multiple passes.
if (operation === constants.BLUR) {
// Treating 'tmp' as a framebuffer.
const tmp = this.getFilterLayerTemp();
// Resize the framebuffer 'tmp' and adjust its pixel density if it doesn't match the target.
this.matchSize(tmp, target);
tmp.draw(() => this._pInst.clear()); // prevent feedback effects here too
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think we don't need either this clear or the image call in line 1090 because we clear in the horizontal pass below. It's not breaking anything, but it's just doing extra work

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was thinking that too. But in the previous code i found the same thing...I also thought to remove but as it was already written I did the same way for framebuffers. Okay gonna remove it.


// setup
Expand Down Expand Up @@ -1130,7 +1132,7 @@ p5.RendererGL = class RendererGL extends p5.Renderer {
});

}
// draw fbo contents onto main renderer
// draw fbo contents onto main renderer.
this._pInst.push();
this._pInst.noStroke();
this.clear();
Expand Down
12 changes: 12 additions & 0 deletions src/webgl/p5.Shader.js
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,18 @@ p5.Shader = class {
* @param {p5|p5.Graphics} context The graphic or instance to copy this shader to.
* Pass `window` if you need to copy to the main canvas.
* @returns {p5.Shader} A new shader on the target context.
*
* @example
* <div class='norender notest'>
* <code>
* let graphic = createGraphics(200, 200, WEBGL);
* let graphicShader = graphic.createShader(vert, frag);
* graphic.shader(graphicShader); // Use graphicShader on the graphic
*
* let mainShader = graphicShader.copyToContext(window);
* shader(mainShader); // Use `mainShader` on the main canvas
* </code>
* </div>
*/
copyToContext(context) {
const shader = new p5.Shader(
Expand Down
2 changes: 1 addition & 1 deletion test/unit/core/rendering.js
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ suite('Rendering', function() {
'camera', 'perspective', 'ortho', 'frustum', 'orbitControl',
'ambientLight', 'directionalLight', 'pointLight', 'lights', 'specularColor', 'spotLight',
'model',
'createShader', 'shader',
'shader',
'normalMaterial', 'texture', 'ambientMaterial', 'emissiveMaterial', 'specularMaterial',
'shininess', 'lightFalloff',
'plane', 'box', 'sphere', 'cylinder', 'cone', 'ellipsoid', 'torus'
Expand Down
Loading