How to use openCV in TypeScript

1

Hi!

I am new to openCV and am trying to apply in a TypeScript project.

I tested it following the instruction on the openCV documentation for using in node.js environment and it worked fine. However, it does not seem to work with the very same code (makes sense since that’s what TypeScript is for).

I found this solution on Github issue section guiding to do npm i mirada which solves type declaration problem in TypeScript.

So if I include console.log('testing..!!') in opencv.js file (the one on the opencv documentation for nodejs), I do see the output string on terminal so I figured that it is imported fine.

My final goal is to warp an image like this here in the last example.

The problems are:

  1. I cannot use cv.imread()
    I keep seeing HTMLImageElement is not definedas an error message on terminal.
    So I used an workaround for reading an image like this
Jimp.read('./lena.jpg')
            .then((lena) => {
                var src = cv.matFromImageData(lena.bitmap);
                let dst = new cv.Mat();
                ...
           
            })
  1. When I console.log()
    const output = cv.warpPerspective(src, dst, M, dsize, cv.INTER_LINEAR, cv.BORDER_CONSTANT, new cv.Scalar());, it does not show or do anything.

If anyone knows how I openCV can be used in TypeScript, please share!

Thanks, everyone!

Happy Coding!

2

the API, in js, seems to read from canvasses and so on, not necessarily files. what exact code did you try to use it in?

3

(no idea about typescript, node, jimp, etc but…: )

  • warpPerspective() does not return anything so there will be no output

  • what is M ?

  • do you receive anything in dst ?

  • try to log / visualize each step in your calculation
    (sure, you even reach that line ?)

  • When I console.log()

    what exactly, how ? please show exact code (your snippet above stops, where things get interesting …)

  • I keep seeing HTMLImageElement is not defined

    so, this is running on a server, not from a browser ?

1 Like
4

Could you share your whole code?

5

What was your parameter when you invoke cv.imread()? I think the information you gave is too short.

6

@David_Jung

there is indeed no imread() (apart from reading from a canvas) in the opencv js api available
(too much work to port all codecs, and js already has a lot of alternative means to acquire images, like the jimp library mentioned above)

7

@crackwitz , that would make sense…! Is this why I keep seeing HTMLImageElement is not defined but then cv.imread() error message? but then is used in the official document (Using OpenCV.js In Node.js).

@berak , oh~! I see I see… alright um the codes that I posted is from the official document instruction for using opencv in node.js (Warp Perspective Transform part).
Also, yes, this is to be used on the server side :slight_smile:

let src = cv.imread('canvasInput');
let dst = new cv.Mat();
let dsize = new cv.Size(src.rows, src.cols);
// (data32F[0], data32F[1]) is the first point
// (data32F[2], data32F[3]) is the sescond point
// (data32F[4], data32F[5]) is the third point
// (data32F[6], data32F[7]) is the fourth point
let srcTri = cv.matFromArray(4, 1, cv.CV_32FC2, [56, 65, 368, 52, 28, 387, 389, 390]);
let dstTri = cv.matFromArray(4, 1, cv.CV_32FC2, [0, 0, 300, 0, 0, 300, 300, 300]);
let M = cv.getPerspectiveTransform(srcTri, dstTri);
// You can try more different parameters
cv.warpPerspective(src, dst, M, dsize, cv.INTER_LINEAR, cv.BORDER_CONSTANT, new cv.Scalar());
cv.imshow('canvasOutput', dst);
src.delete(); dst.delete(); M.delete(); srcTri.delete(); dstTri.delete();

So in a regular JavaScript project, I am able to use the very same code from the Documentation and it works fine (including cv.imread() part).

However, I keep getting errors when I try to apply on a TypeScript project. The first ever issue was “Type” errors and I solved this problem by npm i mirada.

but still have problems (1) reading images with cv.imread() and (2) cv.warpPerspective() parts.

(1) can be somehow solved with Jimp or “maybe” node canvas, but I still wonder how the codes run fine in regular .js files, but not in .ts throwing HTMLImageElement is not defined;

(2) since the document used cv.imshow('canvasOutput', dst);, I figured that the output image is encoded in dst, but I still don’t see any output image

  • I am not actually trying to “save” images as files, but pass it as a variable in a form of ‘base64’ or possible something else.
8
  • What is M?

→ I included the whole code I got from the official document above

  • Do you receive anything in dst?

→ Hmm… not really, but I figured dst has the output image…? Did I get the wrong idea??

You mentioned that imread()is not included in opencv js, but the Documentation has an example using this. Also, it does work in regular node.js project of mine too, but just not with TypeScript :frowning:

I am running all these on server and I guess HTMLImageElement is not defined is the biggest problem right now…

9

correct

the js version is a “mockup”, reading an image from a canvas, you cannot read / decode files from disk

10

@berak , I’m sorry, but what do you mean by JS version is a “mockup”?

I don’t really need to read/decode files right from/to disk. I do have an image drawn on a canvas, but when I pass it to imread(), I keep getting this HTMLImageElement is not defined error :frowning:

11

hey, im NOT A JS DEV (and wont touch node/npm/typescript with a 10 foot pole…)

but maybe you take another look at the js imread() src and try to dissect it.
since you already have a filled canvas, it might boil down to a simple:

function readFromCanvas(canvas) {
    var ctx =  canvas.getContext('2d');
    var imgData = ctx.getImageData(0, 0, canvas.width, canvas.height);
    return cv.matFromImageData(imgData);
};

hmm maybe there should be a check if HTMLImageElement is defined at all ?

12

@David_Lee21 – did above code work for you ?

13

@berak omg! Thanks! This works great! :smiley:
Thank you so much for your help!
I hope this post can be helpful to some other people who are using opencv in TypeScript project.

I guess I’ll keep having fun with opencv from now! Big thank to you again! Thank you!