Hi,
here's how to decode QR code using Unity3D through camera. Please note that I'm using a library called ZXing (Zebra Crossing). I used the C# version since it is the most familiar language to me. I modified it a bit since Unity doesn't allow to use System.Drawing namespace.
To get access to the camera device I use WebCamTexture class which is new since Unity 3.5 (if I'm not mistaken). Here's my code in the Unity side:
using com.google.zxing.qrcode;
This is the only namespace to use from the dll if you just want to decode QR codes and nothing else.
WebCamTexture camTexture; Thread qrThread; void Start () { camTexture = new WebCamTexture(); OnEnable(); qrThread = new Thread(DecodeQR); qrThread.Start(); }
At first I'm declaring a new WebCamTexture to get the texture taken from the camera, and then I'm going to use another thread to process that texture.
int W, H, WxH; void OnEnable () { if(camTexture != null) { camTexture.Play(); W = camTexture.width; H = camTexture.height; WxH = W * H; } } void OnDisable () { if(camTexture != null) { camTexture.Pause(); } } void OnDestroy () { qrThread.Abort(); camTexture.Stop(); }
I use the OnEnable event handler to start the camera and retrieve it's size.
Color32[] c; sbyte[] d; int x, y, z; void Update () { c = camTexture.GetPixels32(); } void DecodeQR () { while(true) { try { d = new sbyte[WxH]; z = 0; for(y = H - 1; y >= 0; y--) { // flip for(x = 0; x < W; x++) { d[z++] = (sbyte)(((int)c[y * W + x].r)<<16 | ((int)c[y * W + x].g)<<8 | ((int)c[y * W + x].b)); } } print(new QRCodeReader().decode(d, W, H).Text); } catch { continue; } } }
Now, everytime a QR code is detected from the camera it will decode it. Nice and simple.
Here is the .dll file: link
Here is the .cs file: link
Thanks.
Thanks for your work, Ariady!
ReplyDeleteYour code is cool,which can implement camera-video decode with best performance.
I've integrade zxing libruary by your post. But did not work out at last.
I've read the origin zxing lib, it seems you have modified some important class like QRCodeReader.
Now I am totally confused about this issue. about zxing. about texture flip.
Could you offer some help?
Or Could you send a copy of you modified zxing lib please?
I'm afraid i can't find your email address. Could you leave an Email to me?
Regards.
Radar
Hi Da L,
Deleteyes I modified the Bitmap parts since Unity does not accept System.Drawing namespace.
The modified Zxing library is provided at the end of my post, here's the link anyway:
http://aurodeus.16mb.com/misc/zxing.zip
If you need the modified version of Zxing Visual Studio project, it is located at:
http://aurodeus.16mb.com/misc/zxingVSproj.zip
I also provided the Unity sample project files:
http://aurodeus.16mb.com/misc/QRunity.zip
Lastly, the texture flipping process is required because the Color32 array from Unity is reversed vertically, meaning that the top most row of the image is located at the bottom most of the array. I hope it's clear enough...
Hey, thank you for your code :)
ReplyDeleteCan you explain me this function?
void DecodeQR () {
while(true) {
try {
d = new sbyte[WxH];
z = 0;
for(y = H - 1; y >= 0; y--) {
Debug.Log ("1. loop");
for(x = 0; x < W; x++) {
Debug.Log ("2. loop");
d[z++] = (sbyte)(((int)c[y * W + x].r) << 16 | ((int)c[y * W + x].g) << 8 | ((int)c[y * W + x].b));
}
}
Debug.Log(new QRCodeReader().decode(d, W, H).Text);
}
catch {
Debug.Log ("failure");
continue;
}
}
}
It is only once in the second loop. Witdh and hight are correct. Where is the failure?
regards,
Oli
Hi Oliver Ebert,
DeleteI'm not sure why it fails. I provide this Unity sample project files:
http://aurodeus.16mb.com/misc/QRunity.zip
The project contains:
a scene, with a main camera, attached the qrCam script file, and the Zxing library itself.
I have tested the project on my laptop in Unity Editor and it runs fine.
I try your code with Vuforia for iOS and Android (Augmented Reality Plugin for Unity) to combine. In your webcam solution is the first pixel left-buttom. On my solution ist the first pixel on right-top. How do I change the code so it works for me?
DeleteSo, do you mean that the array that you got is like:
Delete2 1 0
5 4 3
8 7 6
If that's the case then:
- you don't need to flip it vertically
- but flip it horizontally
So, change the loops in the DecodeQR function to be like this:
for(y = 0; y < H; y++) {
for(x = W - 1; x >= 0; x++) {
...
}
}
Because structure of the array needed for the decode funtion is like:
0 1 2
3 4 5
6 7 8
do you have a chance to test your solution on an iPhone or Android device? I do not get it running. :(
DeleteMaybe I'm wrong but I think the solution can't work properly.
DeleteThe method new QRCodeReader().decode(d, W, H) uses RGBLuminanceSource which requires a sbyte array with red, green and blue values. The code above tries to assign the three channel values as an integer to a sbyte array. The sbyte cast should only return the blue value. The constructor of RGBLuminanceSource inside the method decode should be called with the fourth argument set to true (Is8Bit) and the calculation of the luminance value for the array d should be corrected to something similar like that:
(sbyte)((r + g + g + b) >> 2)
I can't test it myself at the moment because I don't have a running unity installation.
Actually, the bit shifting thing is also copied from the RGBLuminanceSource.cs file (http://code.google.com/p/zxing/source/browse/trunk/csharp/RGBLuminanceSource.cs) (line 108)
DeleteI used that overloaded constructor and not the other (like the one you mentioned) cause I followed the Bitmap "route" (as if giving Zxing a Bitmap data through Unity).
What I did was, I compared the Bitmap array in Visual Studio and the array from Unity using a small 5x6 pixel image file.
And then I chose to follow the "Bitmap route" since I don't know what kind of byte array I should feed Zxing, I mean I don't know the structure of byte array I should give, like the pixel order and stuff. While in the sample app project I can simply see the values of Bitmap array.
reset W,H. because in ios webcam's width&height set when device ready perfect.
Deleteand change for(...)
like this
for(y = 0; y <H; y++) {
for(x = 0; x < W; x++) {
}
it will help u, thanks
I played a little around with unity and the sample source. But I didn't get access to my webcam with unity. Not sure what's wrong.
ReplyDeleteAnyway I changed your code a little bit to show how I would realize it. Perhaps it works and perhaps it helps other people who want to implement a barcode scanner with unity.
https://zxingnet.svn.codeplex.com/svn/trunk/Clients/UnityDemo
It builds without errors but I can't test is because my webcam isn't activated with unity.
Btw. I'm the creator of ZXing.Net which is an up-to-date port of the java original. With version 0.7.0.0 I added a binary for the use with unity which doesn't have a System.Drawing reference.
It would be great if someone can give me a hint how I get my webcam running with unity. Do I need the pro version? Version number is 3.5.2f2.
Oh wow, that's great! I will try your barcode scanner later... Thanks!
DeleteThe WebCamTexture class in unity doesn't need the pro version, I'm using the free version myself.
Have you display the WebCamTexture object somewhere?
Either directly on screen through OnGUI event or display it as a material texture of a plane object..
You're using the very latest unity version right now..
I got it.
DeleteI made some mistakes with namespaces which are not supported for behaviours I think.
You can find my solution here:
https://zxingnet.svn.codeplex.com/svn/trunk/Clients/UnityDemo
It implements a custom luminance source for zxing and uses the BarcodeReader class instead of QRCodeReader. That means it decodes now every supported barcode.
The luminance source calculates the luminance values directly from the Color32 structure. No need to create a temporary byte array.
Works this for Android an iOS?
DeleteNot sure. I didn't try it. But hopefully yes.
DeleteMichael i tested your code where you are using BarcodeReader class instead of QRCodeReader. It works fine for Unity Camera and i am able to decode QRCode using Unity but when i converted it for iOS and droid, it is not working. Can you please help me or anybody else who tried it.
DeletePlease provide some more information because I can't test it on that platforms. What error happens?
Deleteon iOS isn't any error message. Looks like, image isn't recognized or isn't interupted when QRcode is done via var result = barcodeReader.Decode(dummy, W, H);
DeleteI have the same problem.
DeleteI'm not sure if I can give help for iOS and Android because of missing environment.
DeleteBut I added a second sample for unity with vuforia support. Perhaps it would help you.
https://zxingnet.svn.codeplex.com/svn/trunk/Clients/VuforiaDemo
Thx. Downloaded, but looks like missing unity demo scene
DeleteThis comment has been removed by the author.
Delete80472/trunk/Clients/UnityDemo/Assets/UnityDemo.unity on iOS
DeleteDebuger:
ZXing_QrCode_Internal_FinderPatternFinder_find_System_Collections_Generic_IDictionary_2_ZXing_DecodeHintType_object:
.byte 13,192,160,225,128,64,45,233,13,112,160,225,112,93,45,233,56,208,77,226,13,176,160,225,40,0,139,229,44,16,139,229
.byte 44,0,155,229,0,0,80,227,12,0,0,10,44,0,155,229,3,16,160,227,44,32,155,229,0,32,146,229,0,128,159,229
.byte 0,0,0,234
.long mono_aot_zxing_unity_got - . + 304
.byte 8,128,159,231,4,224,143,226,56,240,18,229,0,0,0,0,36,0,139,229,1,0,0,234,0,0,160,227,36,0,139,229
.byte 36,0,155,229,0,0,203,229,40,0,155,229,8,16,144,229,1,0,160,225,0,224,145,229
bl Lm_2d
Is that for me?
DeleteSorry, I'm not sure what I should do with the debugger output.
This comment has been removed by the author.
ReplyDeleteGreat piece of code! I'm having some issues with iOS, though. Textures are flipped twice, and I'm still struggling with that piece of code.
ReplyDeleteI guess it's a minor optimization, buy you could create QRCodeReader just once by adding a member variable
ReplyDeleteQRCodeReader QR;
that is set to null in Start()
and then, in the thread:
if (QR == null)
QR = new QRCodeReader();
output = QR.decode(d, W, H).Text;
being output a string variable that you may use to show the code in a label in OnGUI or something like that. But still struggling to make this work in iOS.
my mobile back camera didnt work this.
ReplyDeletebecause back camera focus-distance high.
how can i handle webcam focus?
sry for my bad english
Right now there is no way to access camera focus using WebCamTexture class from Unity.
DeleteYou could use Vuforia (an AR plugin from Qualcomm for Unity) to access camera autofocus.
You don't need to use WebCamTexture anymore if you're using Vuforia.
thanks about your speed.
DeleteHey -- I know this is a relatively old post now, but I've been trying for awhile to get this to work with Texture2D objects, and I can't get it to work. Even when I generated a code and then plugged it straight back into the decoder, I could not get it to decode -- it always fails. Is there anything about a Texture2D that I'm missing?
ReplyDeleteHere is the class that I'm using:
public class qrCam2 : MonoBehaviour {
public Texture2D encoded;
public string Text = "http://www.google.com/";
void Start () {
encoded = new Texture2D(1024, 1024);
}
void Update () {
// ENCODE //
com.google.zxing.common.ByteMatrix bytematrix = new QRCodeWriter().encode(Text, com.google.zxing.BarcodeFormat.QR_CODE, encoded.width, encoded.height);
Color32[] col = new Color32[bytematrix.Width * bytematrix.Height];
sbyte[] sCol = new sbyte[bytematrix.Width * bytematrix.Height];
int x = 0;
int y = 0;
foreach (sbyte[] sarray in bytematrix.Array){
foreach (sbyte s in sarray){
sCol[1024 * y + x] = s;
col[ ((y)*1024) + x ] = new Color32((byte)s, (byte)s, (byte)s, 255);
x++;
}
x = 0;
y++;
}
encoded.SetPixels32(col);
encoded.Apply();
// DECODE //
try {
print(new QRCodeReader().decode(sCol,bytematrix.Width, bytematrix.Height));
}
catch {
print("fail");
}
}
}
The Texture2D "encoded" displays the proper QR code upside down, so sCol should should have the sbytes in the upside down order as well. But even with plugging that list of sbytes straight into the decode function it doesn't work. Any ideas?
Hmm, the encoded Texture, have you tried to decode it using other app? does it decoded successfully? because I've never tried to encode.
DeleteIn my case, the result I got from Color32[] was inverted vertically, so I reordered the sbyte[] so that the top most row from the original Color32[] would be at the bottom most in the sbyte[].
First of all Thank you for you work and sharing this resource.
ReplyDeleteMyself, I have a funny situation with it. When I run the app it works great - on the firsts decoding (at least in Editor, as I've tested it). It returning value from my QR code in Console, but even when I stop the app (or try to pause it) in Editor the function still returning the value from the QRcode. The camera is off, the Unity editor window is theoretically in stop mode, but it still working and returning value in the Console. Then when I try to play the app in editor again the Unity just hang.
I'll try with your project and I let you know with the results.
Hi,
ReplyDeleteMy test on ipad says: ReaderException was throw and it's about GlobalHistogramBinarizer.estimateBlackPoint(System.Int32[] buckets)
Callstack:
common.GlobalHistogramBinarizer.estimateBlackPoint(Int32[] buckets)
common.GlobalHistogramBinarizer.get_BlackMatrix()
common.HybridBinarizer.binarizeEntireImage()
common.HybridBinarizer.get_BlackMatrix()
BinaryBitmap.get_BlackMatrix()
qrcode.QRCodeReader.decode(BinaryBitmap image, ....)
qrcode.QRCodeReader.decode(SByte[] d, Int32 W, Int32 H)
qrCam.DecodeQR()
Where I should start bugging?
Hi,
Deletefor mobile devices, I don't recommend using this piece of code. It will be very laggy.
Please try the code for Android:
http://ydaira.blogspot.com/2012/09/how-to-decode-qr-codes-using-unity3d.html
I haven't tested it on any ios devices though. But it should be the same. It still uses Zxing.
As for your error message, I'm not sure myself. I'm sorry, can't help there...
On second thought, I remember that screen resolution and camera resolution is not the same, at least on my device. You should throw in camera resolution, not screen resolution. It might be the problem. It may be not, I don't know. Goodluck.
DeleteThank you for fast reply :)
DeleteIt's not problem if it be little laggy. Your better solution proposal was only for Android? My software need to run on ios. Is there way to work around that?
Software is about scanning the qr codes and raporting them to the server. I think that I might do that so that software push .png -files from the camera to the server and then use libdecodeqr (http://trac.koka-in.org/libdecodeqr) from commandline to decode QRs. I also have thought using native code but then I have to do seperated versions for ios, android and standalone.
What you mean throw in camera resolution? I'm using your example code and cameratexture's resolution:
W = camTexture.width;
H = camTexture.height;
Hi,
Deleteyes I wrote there it's for Android, it's because I have not tested it on IOS, but it should run on IOS also, but I can't confirm that. Vuforia plugin also comes in Android and IOS version.
If you still want to use the code from this page, I suggest to take a small portion of the area of image you want to process, because with full camera resolution it would be very slow on mobile devices. Unless you're planning to take a picture and then just process that one image.
Since you're using camTexture width and height it should be okay. I mean, don't give the parameters like this:
W = Screen.Width;
H = Screen.Height;
because screen resolution usually is not the same width camera resolution. Now I have no idea what error you encountered :-?
Sorry for my bad english, I hope it's clear enough :D
hello,
ReplyDeleteZXing DLL file couldn't download
This comment has been removed by the author.
Deletehttps://www.dllme.com/download.php?file=10723&id=b15314553420097274c6d86e667b3b86&key=ec658366ecebc643bbd5db62e27effbc28ee707dfbbf7ce832dd7b40f0101088&iss=1567797410
Delete