A curated, though not necessarily comprehensive, list of projects I've worked on professionally, academically, and on my own time. Click on any project to read more about it.
A curated, though not necessarily comprehensive, list of projects I've worked on professionally, academically, and on my own time. Click on any project to read more about it.
Using OpenCV-Python, scikit-image, and scikit-learn, I extracted histogram of oriented gradients (HOG) features, as well as color histogram and 2D spatial features, from images of vehicles and non-vehicles from the KITTI and GTI datasets to train a linear support vector machine (SVM) classifier. To improve the accuracy of the classifier, I determined the most effective color space and combination of color channels to use for classification, as well as the optimal parameters and tolerances for the classifier itself.
I employed a sliding window search to find cars in an actual video, limiting the search to the region below the horizon and increasing the size of the sliding window as as it approached the bottom of the frame, since vehicles closer to the bottom are closer to the camera and, hence, appear larger.
This produced a set of detections for any given frame. Actual cars (as opposed to false positives) yielded numerous detections.
To determine which regions of the image represented a vehicle (and to combine multiple detections of the same object), I computed a heatmap of the detections. To reduce the effect of transients in any given frame, I summed the heatmaps of the last several frames and ignored pixels in the heatmap below a certain threshold.
Next, I applied simple connected component analysis to the heatmap to identify and distinguish individual vehicles.
Finally, I drew bounding boxes around objects larger than some minimum size to produce the final video at the beginning of this description. While most contemporary object detection schemes utilize neural networks, completing this project provided me a better understanding of various image recognition techniques, as well as insight into both the advantages and shortcomings of SVMs.
While working at Quantum Rehab, I was tasked with designing a prototype for a new articulating foot platform (AFP), or power leg-rest, for use on Quantum power wheelchair TRU-Balance seating systems. The current AFP is pictured above, both retracted (top) and extended (bottom). The goal was to create a smaller and lighter design to reduce the overall weight of the power chair without compromising performance.
To aid in the design of a new prototype, I first developed a kinematic model of the four-bar mechanism utilized by the existing model, pictured below.
Furthermore, to quantify the effect of the device on the legs of users, I developed a simplified inverse kinematic model of the human leg (below).
Combining the kinematics of the device and the leg with the weight of each component, I developed a dynamic model of the system, with the aim of solving for the axial forces acting on the linear actuators in the device.
I wrote Python scripts to numerically solve for these forces over the device's range of motion. To verify the accuracy of the model, I performed real-world tests of the AFP under a variety of loading conditions. The model correctly predicted the behavior of the actual device with >95% accuracy.
Having verified the model, I performed sensitivity analysis to optimize and modify key parameters of the design, allowing me to develop a late-stage prototype that was 25% lighter and 15% slimmer than the existing model, and whose range of motion was 10% greater than the existing model. Furthermore, since the AFP sits at the front of the wheelchair, the weight reduction contributed to the stability of the chair by shifting its overall center of gravity farther back.
I also set up and secured the VPS—a DigitalOcean droplet—on which this site is currently hosted, which involved installing and configuring Apache, MySQL, and PHP, among other things.
Read more about this project on my blog.
For this exercise in vehicle dynamics, I numerically solved the differential equations of motion of a half-car model using the Euler method, then animated the result in Python with matplotlib. A half-car model considers half of a car and treats both the suspension and the wheels as springs and dampers.
This is a 4-DOF system in which the vertical displacement of each wheel, the vertical displacement of the chassis, and the pitch angle of the chassis may vary.
An advantage of numerical solutions, like the Euler method, is that they make it relatively easy to solve for the response of the system with nonlinear inputs, at the potential cost of stability and accuracy.
The vehicle outline is that of a 2010 Honda Accord Coupe—a car that's treated me well over the years.
Being mildly colorblind myself, I thought it would be interesting to utilize K-means clustering, a conventional machine learning technique, to distinguish the numbers in Ishihara color blindness tests from the backgrounds.
I wrote a command-line Python utility that facilitates color-based segmentation of an image by converting the image to a desired color space (e.g., HSV, Lab, etc.), selecting any combination of color channels on which to perform the K-means clustering, selecting the number of clusters for the algorithm, and writing the output to the disk in the form of an image file.
Examples of the output of the tool, which utilizes scikit-learn and OpenCV, are shown above for one particular Ishihara plate using different color spaces, color channels, and numbers of clusters.
As part of a mechatronics course at Purdue, I worked with a team to develop a small autonomous robot charged with performing the following tasks without human intervention:
Our design called for infrared distance sensors to detect both the softballs and the test area walls, and infrared line-following sensors to find and follow the black line. The ball-collection mechanism was essentially a "claw" or inverted scoop attached to a motor that, with the help of additional motorized rollers, coaxed each softball into a channel. An early SolidWorks model of the design can be seen below.
The walls were not attached directly to the floor, but were several inches above the floor, as seen below.
We exploited this fact by placing one set of IR distance sensors close to the floor and another set higher up on the robot to distinguish walls from softballs. These sensors, as well as the ball-collection mechanism, can be seen in the figures below.
For control, we developed a finite state machine and wrote code in C that was deployed on an Arduino Mega. We also made use of existing C++ libraries for tasks related to line following.
In the end, the robot was moderately successful, though not without its hiccups. If I were to tackle a similar problem again, I would likely leverage the ability of a single camera and computer vision techniques, perhaps running on a Raspberry Pi, to replace the numerous IR sensors.
Find the code on Github.
Image segmentation based on color thresholding is relatively common in computer vision applications. Color-based object tracking is a notable example. However, determining the range of color channel values between which a specific object or feature falls can be a hassle.
Using OpenCV, I wrote a utility to allow a user to rapidly segment a video or image by adjusting the lower and upper values for each color channel and visualizing the result in real time. I wrote both a C++ version of the program and a Python version.
Find the code on
Github (in the
read more about the code and the implementation in
this blog post, or read more about the mathematics
and derivation of the Jacobian inverse method in
this series of blog posts.
To visualize the Jacobian inverse method for solving the inverse kinematics of a system in two dimensions, I wrote a Python program that made use of numpy and matplotlib to simulate a robot arm consisting of an arbitrary number of linkages connected by revolute joints.
In the video above, the linkages are represented by green line segments, the joints by solid green circles, and the single end-effector by a hollow circle. The robot arm is made to track a target represented by a red dot. The dashed circle represents the maximum reach of the robot arm, based on the sum of the lengths of the individual linkages. In the video, this is used to demonstrate a limitation of the Jacobian inverse method: its instability at singularities, which occurs when all of the joints are collinear.
The matplotlib plot is interactive and contains two modes. In the first mode, the user can click anywhere within the plot to place the red dot, after which the robot arm automatically tracks it. With a keypress, the user can switch to a second mode in which the dot travels along a pseudorandom path, demonstrating the ability of the robot arm to track a moving target.
While learning about graph traversal and graph-based path planning methods, I wrote a Python script that utilized an interactive matplotlib plot to animate the breadth-first search method for finding the shortest path between two points on a grid.
As the video above demonstrates, the program randomly places a start cell (green), a destination cell (red), and obstacles (black), then illustrates the expanding wavefront of visited cells with blue. If the destination cell is found, the algorithm backtracks to display the shortest path, or one of the shortest paths, if multiple solutions exist.
The program, being interactive, allows the user to adjust the number of grid rows and grid columns, adjust the obstacle density (probability that any given cell will be an obstacle), as well as randomize the locations of the start cell, destination cell, and obstacles.
MS thesis title:
The effect of body weight support on squat biomechanics
Link to abstract
My Master's thesis combined robotics and biomechanics, the goal being to investigate the effect of providing partial weight support on the forces and moments in the ankle, knee, and hip joints.
The squat and squat-like motions, such as the sit-to-stand and stand-to-sit transitions, are ubiquitous among all people in all parts of the world. Partial weight support is often used in physiotherapeutic settings for the physical rehabilitation of individuals, and assistive or augmentative devices like robotic exoskeletons also provide a form of partial weight support. While the effect of weight support on fundamental motions like walking and running has been well studied, there have been few or no studies on how weight support affects the squatting motion. My thesis aimed to help fill this gap.
I sought to characterize the effect by quantifying the forces and moments in the joints of the lower limb via inverse dynamics, which involves measuring the ground reaction force (GRF) and the positions of the lower limb segments to back-calculate the reactions in each joint based on the position and inertial properties of each limb segment. The free body diagrams above illustrate the technique.
The experimental setup is illustrated in the figure above. To measure the ground reaction force, users performed squats on a force plate–instrumented treadmill (the treadmill wasn't running—it was only used for its force plates). Weight support was provided by means of a harness connected via ropes to pneumatic actuators.
As mentioned above, ground reaction forces were obtained with force plates. The applied force was measured by load cells. The user's motion was recorded with Vicon infrared cameras, shown in the figure above. The figure below shows the markers, which were placed on the subjects' joints and on various points on their legs, as seen by the cameras.
The pneumatic actuators were connected to a National Instruments cRIO embedded controller. To control the actuators, provide a consistent amount of weight support, and automate the data acquisition process, I wrote custom LabVIEW VIs and implemented a PID control loop, allowing the system to conduct trials on its own, informing the user when to prepare for the next set or the next repetition, and waiting until the user was ready before beginning to apply the desired amount of weight support.
To process the raw data and perform the inverse dynamic analysis, I wrote a number of Matlab scripts, the result of which can be seen in the figure above. I also developed a simplified model of the weight-supported squatting motion, illustrated in the diagram below, with which to compare the experimental results.
By running trials at several amounts of weight support and comparing the results by means of ANOVA and post-hoc t-tests, I found that, as expected, the joint moments decreased relatively proportionally with weight support. The more interesting result, though, had to do with the kinematics of the squatting motion. For most of the squatting motion, the kinematics of the supported squat and unsupported squat were similar. However, about 3/4 into the rising portion of the squat, the kinematics of the supported squat diverged considerably, with the users consistently almost "lurching" forward, a fact that was also reflected by the center of pressure of the ground reaction force shifting forward.
The takeaway was that, while weight support predictably alleviates some of the load on the joints during a squat, it can also significantly alter the normal, natural motion of the body. The true cost of these changes to the body's natural biomechanics is unknown—they may be harmless or they may be detrimental. In fact, studies of assistive robotic exoskeletons have found that, even though the exoskeletons ostensibly reduce the load on the body, they oftentimes increase metabolic cost and oxygen consumption, suggesting that the person actually has to work harder with support to compensate for the unnatural movement.
In other words, seemingly straightforward actions can have unforeseen and poorly understood consequences. Any device that affects the body's natural biomechanics must be carefully investigated, even if it seems, intuitively, like it ought to be purely beneficial.
In the power wheelchair industry, the term "mid-wheel drive" refers to a class of wheelchairs possessing six wheels, with the middle wheels being attached to the drive motors. The weight distribution of a power wheelchair (the load borne by each wheel) significantly affects its stability and performance, which are paramount for both user safety and comfort. The ability to analytically model, characterize, and manipulate the weight distribution of a power wheelchair can, therefore, be extremely valuable.
However, a vehicle with six wheels is statically indeterminate, meaning that the normal forces of the wheels on the ground cannot be calculated. Even if we assume the vehicle and weight distribution are symmetric and consider only half the vehicle, if the three wheels lie in the same plane, the system is still indeterminate. This problem is illustrated below.
The traditional solution to this problem involves incorporating the stiffness of the wheels, treating each wheel as a spring. By assuming the deflection of each wheel is equal (i.e., the vehicle remains level), the normal forces can be computed, as shown below.
In the case of the Quantum Edge 2 mid-wheel drive power wheelchair, this approach didn't work—actual measurements of the weight distribution (normal forces) did not match those predicted by this simple model, likely due to the complex and nonlinear relationships between the components of the suspension. Essentially, the suspension components of the different wheels are directly linked. This particular system is not only statically indeterminate, it's also kinematically indeterminate—its geometric configuration cannot be determined without knowing the normal forces (which affect both tire compression and deflection of the suspension springs), but the forces cannot be determined without already knowing its geometric configuration. It is a Catch-22.
To address this, I developed a 5-DOF potential energy model of the system in which the compression of each tire, the height of the base of the power chair, and the pitch angle of the base were allowed to vary. The animation below shows how the compression of the springs in the suspension system changes as the height of the base is varied (for this simplified demonstration, the compression of each tire is held constant, as is the pitch angle of the base).
The animation was made in Python with matplotlib. I also used Python to compute the potential energy of the system in any given configuration (i.e., any given set of values for tire compressions, base height above the ground, and base pitch angle). The potential energy equation for the system, which takes into account the gravitational potential energy of the components of the wheelchair and the elastic potential energy due to deflection of the springs and tires, is shown below.
With the aid of scipy multivariate numerical optimization methods, I used this function to find the configuration of the system that minimized the potential energy, exploiting the fact that a system will tend toward the state in which its potential energy is lowest. This principle, which, in physics and thermodynamics, is known as the minimum total potential energy principle, is commonly used in solid mechanics to solve finite element structural models. The plot below shows how the overall potential energy of the system varies with ground clearance, allowing us to determine the ground clearance at which potential energy is minimized (for the case where the parameters corresponding to the other degrees of freedom are held constant).
My model and solution correctly predicted the ground clearance of the actual Quantum Edge 2 power wheelchair with an accuracy of >95%, and correctly predicted the normal forces to within 80% of the measured values, providing a better understanding of the properties of an otherwise obscure suspension mechanism.
The video above demonstrates the use of a tool I made in Python that, by combining OpenCV and matplotlib, computes and displays the three color channel histograms of a video in real time. The user can also choose to display the video feed in grayscale and display the single grayscale color channel histogram. The number of histogram bins can be customized as well.
For part of a larger project at Quantum Rehab, I found myself having to design a linear actuator from the ground up, as no existing off-the-shelf solutions met the requirements for our particular application, which had numerous unconventional space restrictions.
I first familiarized myself with the basic principles of leadscrew-based linear motion systems (the free body diagram above is a simple visual example of one component of this process), and used these principles to derive the equations of leadscrew frictional torque specific to our application.
Part of designing an ideal linear actuator for the application involved selecting a motor whose stall torque and no-load speed would allow the actuator to complete its motion in a reasonable amount of time. To that end, I developed a set of equations with which to compute the optimal motor characteristics for the application.
The equation above permits us to obtain the set of actuator no-load speeds (vNL) and stall loads (Fstall) that would enable an actuator to travel its entire stroke length smax within a time T, given a function F(s), which provides the force on the actuator (the force it must overcome) as a function of the stroke length s (note that F(s) depends on the particular application, and is related to the kinematics and dynamics of the system in question). Numerically solving this equation for a range of stall loads Fstall and plotting the result yields the following curve for a desired time T and a hypothetical function F(s) (for brevity, F(s) is not shown here):
In the figure above, which I created using matplotlib, the solid black line represents the set of (Fstall, vNL) values that would allow the actuator to travel its full stroke length in time T—in this case, 18 seconds. Actuators with values in the green region above the curve would complete the motion faster than T and actuators with values in the yellow region beneath the curve would complete the motion more slowly than the desired time T. An actuator with values in the red region would be unable to complete the motion, as the stall load of the actuator would be lower than the maximum load that the actuator would face over the course of the motion.
The same principle can be applied directly to a motor in a linear actuator by considering the torque τ instead of the force (i.e., axial load) F, the angular displacement θ instead of the stroke (i.e., linear displacement) s, and the angular no-load speed ωNL instead of the linear no-load speed vNL:
Having developed these equations, I determined how the parameters would change at different gear ratios (taking into account how inertia is nonlinearly reflected through the leadscrew and gear train to the motor), and wrote a Python script to plot curves like the one above at different gear ratios N.
This provided a solid guideline for selecting both a motor and gearing. In addition to the motor and gearing, my script also helped optimize the leadscrew parameters (pitch, lead, lubricant friction coefficients, and resultant self-locking ability), end fixity to increase the leadscrew critical speed and buckling load, and other factors.
Ultimately, this analytical approach allowed me to design a novel prototype for a new type of linear actuator.
This was a simple project in which I implemented several common sorting algorithms and demonstrated, in Python, how to use the matplotlib animation module with generator functions to animate the algorithms and count the number of operations (swaps, comparisons, etc.) required to sort a list of integers.
Conway's Game of Life is a well-known "cellular automaton," or "game" in which each cell of a 2D grid can be either alive (on) or dead (off). The grid is iteratively updated, with each iteration referred to as a generation. The original rules state that 1) any living cell with exactly two or three neighbors survives to the next generation (otherwise, it dies), and 2) any dead cell with exactly three neighbors becomes alive during the next generation, as if born by reproduction.
This can make for some interesting behavior and unique patterns, a few examples of which are shown in the video above and in the animations below.