Virtual Mouse Project

5 次查看（过去 30 天）
Syahrul Fikri
Syahrul Fikri 2012-3-16
评论： Aakash 2015-1-13
Hey everyone, I'm doing a virtual mouse project by controlling the mouse pointer on the computer screen using a webcam to track movement of my hand n some mouse clicking event. I have to learn matlab from scratch n I've tried doing as possible as I can but still can't produce the right codes or thru simulink. Can anyone really help me on this bcuz I'm kinda desperate right now n I'm really don't know what I'm suppose to do next. If anyone can give me the codes or simulink models, that would be really helpful, thank u..
  7 个评论
显示 5更早的评论
Syahrul Fikri
Syahrul Fikri 2012-4-12
final step people!! how do i convert my virtual mouse m.file to exe.file?? i tried using the deployment project method - file/new/deployment project.. , all i got is error - >> coder('-build', 'Virtual Mouse (Red Object Tracking).prj') ??? This text contains non-empty top-level expressions. It appears to be a script. Error in ==> tracking_red Line: 1 Column: 1 Code generation failed: Open error report. - my codes are below for checking
Walter Roberson
Walter Roberson 2012-4-12
You can only compile function .m, not script .m . Add a "function" line to your .m file.

请先登录，再进行评论。

请先登录，再回答此问题。

回答（9 个）

Matt Kindig
Matt Kindig 2012-4-4
Controlling the mouse pointer is actually a bit difficult, as you need to use the Java Robot class (it's not a Matlab function per se). I'm not even sure how you would access this via Simulink.
See below: http://www.mathworks.com/support/solutions/en/data/1-2X10AT/index.html?solution=1-2X10AT
  1 个评论
Walter Roberson
Walter Roberson 2012-4-4
Example Java Robot use of keyboard: http://www.mathworks.com/matlabcentral/answers/5259-how-to-simulate-keyboard-key-inputs
The Solution that Matt shows above has an example of mouse control.
You could use a MATLAB Function block to access this kind of code; you would create the robot for the initialization phase and give commands to the robot based upon the run-time input signals.

请先登录，再进行评论。

Teja Muppirala
Teja Muppirala 2012-4-4
MATLAB can move the mouse around:
x = 500;
y = 600;
set(0,'PointerLocation',[x y]);
But, I think you do need the Java Robot to make a click.
  1 个评论
Walter Roberson
Walter Roberson 2012-4-4
Unless you can figure out which object you are over and locate its callback function and activate it, passing in a suitable "event" structure to represent right-click, shift-click etc.

请先登录，再进行评论。

Syahrul Fikri
Syahrul Fikri 2012-4-7
hye all, ive found this one m file code from the matlab central file exchange, this code is to track red object in real time image acquisition, my question is, how do u connect the tracking bounding box to the mouse input from the java robot library?? the following are the codes
% Capture the video frames using the videoinput function % You have to replace the resolution & your installed adaptor name.
vid = videoinput('winvideo',1,'RGB24_320x240');
% Set the properties of the video object
set(vid, 'FramesPerTrigger', Inf); set(vid, 'ReturnedColorspace', 'rgb') vid.FrameGrabInterval = 5;
%start the video aquisition here
start(vid)
% Set a loop that stop after 100 frames of aquisition
while(vid.FramesAcquired<=200)
% Get the snapshot of the current frame
data = getsnapshot(vid);
% Now to track red objects in real time
% we have to subtract the red component
% from the grayscale image to extract the red components in the image.
diff_im = imsubtract(data(:,:,1), rgb2gray(data));
%Use a median filter to filter out noise
diff_im = medfilt2(diff_im, [3 3]);
% Convert the resulting grayscale image into a binary image.
diff_im = im2bw(diff_im,0.18);
% Remove all those pixels less than 300px
diff_im = bwareaopen(diff_im,300);
% Label all the connected components in the image.
bw = bwlabel(diff_im, 8);
% Here we do the image blob analysis.
% We get a set of properties for each labeled region.
stats = regionprops(bw, 'BoundingBox', 'Centroid');
% Display the image
imshow(data)
hold on
%This is a loop to bound the red objects in a rectangular box.
for object = 1:length(stats)
bb = stats(object).BoundingBox;
bc = stats(object).Centroid;
rectangle('Position',bb,'EdgeColor','r','LineWidth',2)
plot(bc(1),bc(2), '-m+')
a=text(bc(1)+15,bc(2), strcat('X: ', num2str(round(bc(1))), ' Y: ', num2str(round(bc(2)))));
set(a, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'yellow');
end
hold off
end
  2 个评论
Walter Roberson
Walter Roberson 2012-4-7
x = bc(1);
y = bc(2);
and then use the technical solution Matt referred to.
Syahrul Fikri
Syahrul Fikri 2012-4-7
the code works!!! thank u so much for all your help walter, im truly grateful, thanx a zillion times

请先登录，再进行评论。

vaibhav mathur
vaibhav mathur 2012-4-7
put the value of red object's centroid in the [x y] mat. and set(0,'PointerLocation',[x y]); and the pointer moves according to centroid may be it work good luck!
Syahrul Fikri
Syahrul Fikri 2012-4-7
hye all, i would like to say thank u so much for all your help n suggestions, finally, my code works by using the red tracking code n connect it to the java robot, thanx again all, if i encounter further problems, ill post it here, :D
Syahrul Fikri
Syahrul Fikri 2012-4-8
Hye all, from the code above, I've successfully execute it with the java robot command, I hv a question, can anyone explain to me how can a mouse clicking event being execute when the program detects 2 tracking red object, the code can detect 2 red object, my plan is the first red object is to move the mouse pointer, but when it detects the second red object, a mouse clicking event is execure
  4 个评论
显示 2更早的评论
vidit virmani
vidit virmani 2013-2-19
hey.... i feel lucky to find this page and the above discussion.... because i am too working on the same project of controlling mouse by color detectin methods.... i used the same code to detect a color.. i used blue instead of red.. i planned to execute the clicking events by manipulating the distance calculated between my finger and the thumb(both covered with a blue tape).. how can i do that....?? . . %% import java.awt.Robot; import java.awt.event.*; mouse = Robot; mouse.mouseMove(bc(1), bc(2)); mouse.mousePress(InputEvent.BUTTON2_MASK); mouse.mouseRelease(InputEvent.BUTTON2_MASK); %% do i have to use this code directly with the color detection code(as u have stated in above comment) or some prerequisites are to b done to interface java with MATLAB..?? please guide me for i am a new matlab user...
Aakash
Aakash 2015-1-13
Hi, could you please tell me how the clicking is done here...please...!! thanks..!

请先登录，再进行评论。

vidit virmani
vidit virmani 2013-2-21
@syahrul fikri : i used your code and its working... the mouse pointer moves and also clicks, but the problems i faced are..... 1) i dont know how the clicking operation is being executed.... does it executes when pointer is stable on a folder for some specified time or when another color is detected by the system?...... 2) if i move my hand to right , the pointer moves left and vice versa..... how to reverse that....... 3)if the camera resolution is less than the resolution of the screen, whole screen is not covered with this system.... how to eliminate this?..... 4)if i want to detect two different colors and execute clicking operations by manipulating the distance between the colors, how can i do that..?? please help....
  1 个评论
Rajshekar
Rajshekar 2013-4-24
@vidit: 2) this is normal mirror concept: hence the pointer position shall have the formula of [1024-x, 780-Y] where x and y are the original positions in your code. the numbers 1024 AND 780 SORT OF RESOLUTION OF YOUR SCREEN.It depends on your screen resolution.

请先登录，再进行评论。

Rajshekar
Rajshekar 2013-4-24
Hey i tried to check this interesting thing: i used java robot.was able to move pointer and click as well. But there is this thing : when i right clicked a block in simulink(ofcourse by java robot), menu drop down appeared as normal; in the next lines of codes, i meant to position cursor on one of the menu item so i tried to reposition and use right click to select the menu item. But by the time these lines were to execute, the menu was not visible and hence the menu item cannot be selected
i think the menu items disappears when next lines of codes starts execution because the script window activates.
Do you know how to avoid this disappearance of the menu.
cpe111
cpe111 2014-12-5
Hey can you help me, please ? my project is how to control mouse by face , I have done face with nose and eyes detection , but I don't know how to control the cursor movements and clicking ! any help please ! because my matlab knowledge is so weak !
Thank you .

类别

在 Help Center 和 File Exchange 中查找有关 Startup and Shutdown 的更多信息

标签

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by