
Optical tactile array with per-taxel shear sensing
A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning


Low-cost tactile arrays have made touch a practical input for robot learning, but piezoresistive designs sense only pressure, while camera-based sensors that also resolve shear are too bulky to tile over a gripper. To solve this problem, we present LuxTac, a flexible, scalable optical taxel array, about 3.5 mm thick and built from off-the-shelf parts. Each taxel is a digital RGBC light-to-digital converter under a translucent white silicone layer capped by an opaque black film, so the back-scattered LED light falls monotonically with local indentation depth.
The same board accepts two interchangeable gel cartridges: a plain one for normal sensing, and one with colored walls that encodes the local shear in the color of the returned light, with no camera, magnets, or bump overlay, making LuxTac, to our knowledge, the first optical taxel array to resolve shear at individual taxels. Characterized with an automated CNC indenter and a six-axis force/torque sensor, the normal cartridge responds linearly to force (R² = 0.999) from 104 mN to at least 5.8 N, with 2.0% repeatability, 5.9% hysteresis, and a taxel-to-taxel variation of 6–10% after normalization. With the shear cartridge, per-taxel models estimate the three-axis force at every taxel; across three indenter shapes and nine held-out sessions, the summed estimates reach R² = 0.96 / 0.96 / 0.92 in x / y / z, each taxel’s estimate 0.92 / 0.93 / 0.94, and the median shear-direction error is 4.8°. Adding LuxTac pressure readings to an Action Chunking Transformer policy substantially improves peg-in-hole success over a vision-only baseline. Design files, code, and videos will be available on this page.
Each taxel consists of a photodetector and nearby LEDs under a white translucent silicone layer, capped by a black opaque layer. LED light entering the white translucent silicone is scattered; part of it returns to the detector, and part reaches the black opaque layer and is absorbed. The black opaque layer also shields the detector from ambient light.
Indenting the surface reduces the local thickness and therefore reduces the returned light. A key consequence is that the raw reading decreases with load and depends on the deformation of the elastomer, not on force directly.
In the shear-encoded gel, four-color silicone walls separate the white translucent cubes above the taxels. Shear drags the colored walls sideways, so each detector sees a different mixture of colored light: a detector reports normal deformation through its brightness and shear through its chromaticity.
LuxTac comes in two modes: two interchangeable gel cartridges on the same board and electronics. They share the same materials and base process, and differ in the structure of the white translucent layer.
Each converter integrates an IR-blocking filter, a photodiode array, and a 16-bit integrating ADC per channel, so the analog signal never leaves the package. All converters share one fixed I²C address, so two TCA9548A switches give each sensor its own bus segment with dedicated 4.7 kΩ pull-ups.
| Mode 1: intensity-encoded gel | Mode 2: shear-encoded gel | |
|---|---|---|
| Senses | Normal deformation | Normal deformation and shear |
| Channels read | Clear (C) only, one value per taxel | All four RGBC, four values per taxel |
| White layer | One continuous white translucent layer, 21.5 × 21.5 mm cavity | One white translucent cube per taxel (about 3.8 × 3.4 mm), 16.4 × 18.0 mm cavity |
| Color | None | Four-color silicone walls in 1 mm slits: rows alternate green / red, columns yellow / blue |
| How it reads | Pressing thins the layer, so less light returns | Brightness for normal deformation, chromaticity for shear |
| Stack | Black opaque layer, white translucent layer, soft acrylic layer, flexible PCB | Black opaque layer, white translucent layer, four-color silicone walls, soft acrylic layer, flexible PCB |
| Mold | 0.5 mm acrylic base, 2 mm acrylic frame, heat-set inserts, M2 screws | Same, plus 3 column walls and 3 row walls with 1.25 mm half-lap notches |
| Stiffness | kn = 3.86 ± 0.09 N/mm per taxel (normal) | 1.01 / 1.25 N/mm at 0.5 / 1.5 mm depth (shear) |
Every taxel is an independent digital detector on its own bus segment, so the same taxel cell tiles into arrays of other sizes by adding sensors and multiplexers, without changing the readout scheme.




Faster Cheaper Better
1,024 taxels · design
With three-axis force at each taxel.
Contact us for the design filesComing soon
We are building a platform where you can design your own LuxTac PCB: pick a size and layout, generate the board, and build it yourself.
After normalization, every taxel responds linearly to force (R² = 0.999).
Each taxel estimates its own three-axis force, across three indenter shapes and nine held-out sessions.
| Metric | Value |
|---|---|
| Normal cartridge | |
| Noise σ | 1.06 ± 0.44 counts (0.08% of baseline) |
| Detection threshold | 104 ± 49 mN |
| Working range | 0.10 to 5.8 N (saturation not reached) |
| Sensitivity, linearity | 0.170 FS/N, R² = 0.999 |
| Hysteresis | 5.9 ± 1.1 %FS |
| Repeatability | 2.0 ± 0.3 %FS |
| Crosstalk, −x / ±y neighbor | 0.7 ± 1.2 / 5.2 ± 6.0 % |
| Crosstalk, diagonal neighbor | 0.8 ± 1.6 % |
| Per-taxel stiffness kn | 3.86 ± 0.09 N/mm |
| Force from h ∘ g−1, RMSE | 0.20 N (R² = 0.988) |
| Shear cartridge | |
| Force R², x / y / z, unseen positions | 0.85 / 0.75 / 0.90 |
| Force R², x / y / z, held-out session | 0.96 / 0.96 / 0.92 |
| Per-taxel force R², held-out session | 0.92 / 0.93 / 0.94 |
| Shear direction error, unseen positions (circle) / session | 14° / 4.8° |
| Contact location error, unseen positions | 0.56 mm |
| Force repeatability between sessions | 0.05–0.10 N |
| Normal linearity R²; noise | 0.996; 0.10% of baseline |
Positions never seen in training, replayed on the CNC bench: LuxTac per-taxel forces vs. the ATI force sensor.
Held-out position T01, 3× speed.
Held-out position T06, 3× speed.
We simulate LuxTac in IsaacGym with TacSL, on a Franka Panda arm and parallel-jaw gripper, as in the real comparison grasps. Each taxel's penetration depth is turned into the reading the real taxel would give, using the depth curve fitted on the calibration bench, so no raw counts, baselines, or LED drift have to be simulated.
Over 12 grasps of four 3D-printed objects in three orientations, simulated and real contact patterns agree with a mean cosine of 0.942, against 0.59–0.76 for the wrong object, orientation, or taxel order.
A simulated grasp in IsaacGym. Top right: the simulated LuxTac reading.
| Real frames compared against | Score |
|---|---|
| Real frames of the same hold (ceiling) | 0.9994 |
| Matched simulation | 0.942 |
| Uniform pad | 0.763 |
| Simulation of a different object | 0.674 |
| Same object, different orientation | 0.642 |
| Matched simulation, taxels shuffled | 0.594 |
| 45° grasps: correct / mirrored diagonal | 0.946 / 0.264 |
| One-to-one assignment, object recovered | 10 / 12 |
| Wasserstein-1 distance of pooled values | 0.067 |
Action Chunking Transformer policies, vision only vs. vision + LuxTac, on the two manipulation tasks.
LuxTac raises peg-in-hole success from 6/15 to 13/15.
Vision + LuxTac clips show the live LuxTac reading in the bottom-left inset,
frame-synced to the camera 0 → 1.
| Task | Vision only | Vision + LuxTac |
|---|---|---|
| Peg-in-hole | 6/15 (40.0%) [20–64] | 13/15 (86.7%) [62–96] |
| Cube sorting | 5/15 (33.3%) [15–58] | 6/15 (40.0%) [20–64] |
| Average | 36.7% | 63.3% |
@misc{prince2026luxtacflexiblescalableopensource,
title={LuxTac: A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning},
author={Md Rakibul Islam Prince and John Pelan and Zhiyuan Zhang and Adyaa Khera and Katherine Pesetski and Adeesh Desai and Akimizu Sugiyama and Minwoo Ki and Benjamin Alan Capelle and Temitope Ibrahim Amosa and Zijie Xu and Yihan Huang and Taehun Chung and Long Wang and Wei Li and Shaoting Lin and Yu She},
year={2026},
eprint={2611.23575},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2611.23575},
}