Optical tactile array with per-taxel shear sensing

LuxTac

A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning

Md Rakibul Islam Prince John Pelan Zhiyuan Zhang Adyaa Khera Katherine Pesetski Adeesh Desai Akimizu Sugiyama Minwoo Ki Benjamin Capelle Temitope Ibrahim Amosa Zijie Xu Yihan Huang Taehun Chung Long Wang Wei Li Shaoting Lin Yu She
LuxTac normal cartridge: black opaque gel in the mold on the flexible PCB.
Normal cartridge
LuxTac shear cartridge, backlit so the four-color silicone walls glow through the gel.
Shear cartridge
Scalablesame cell tiles to 8×4, 8×8, …
R² = 0.999linear to force
104 mNdetection threshold
63.3% vs 36.7%average robot success, with vs without touch

Abstract

Low-cost tactile arrays have made touch a practical input for robot learning, but piezoresistive designs sense only pressure, while camera-based sensors that also resolve shear are too bulky to tile over a gripper. To solve this problem, we present LuxTac, a flexible, scalable optical taxel array, about 3.5 mm thick and built from off-the-shelf parts. Each taxel is a digital RGBC light-to-digital converter under a translucent white silicone layer capped by an opaque black film, so the back-scattered LED light falls monotonically with local indentation depth.

The same board accepts two interchangeable gel cartridges: a plain one for normal sensing, and one with colored walls that encodes the local shear in the color of the returned light, with no camera, magnets, or bump overlay, making LuxTac, to our knowledge, the first optical taxel array to resolve shear at individual taxels. Characterized with an automated CNC indenter and a six-axis force/torque sensor, the normal cartridge responds linearly to force (R² = 0.999) from 104 mN to at least 5.8 N, with 2.0% repeatability, 5.9% hysteresis, and a taxel-to-taxel variation of 6–10% after normalization. With the shear cartridge, per-taxel models estimate the three-axis force at every taxel; across three indenter shapes and nine held-out sessions, the summed estimates reach R² = 0.96 / 0.96 / 0.92 in x / y / z, each taxel’s estimate 0.92 / 0.93 / 0.94, and the median shear-direction error is 4.8°. Adding LuxTac pressure readings to an Action Chunking Transformer policy substantially improves peg-in-hole success over a vision-only baseline. Design files, code, and videos will be available on this page.

Key Highlights

Scalable gridTCS34725 RGBC converters on a 4.42 × 4.80 mm pitch, LEDs between the taxels, four-layer flexible PCB.
3.5 mm thin, flexibleThe whole sensing pad is only 3.5 mm thick, flexible PCB included. No camera, no analog front end.
Shear at every taxelColored walls encode shear in the color of the returned light: three-axis force at each taxel with R² = 0.92 / 0.93 / 0.94 (x / y / z) and a 4.8° median direction error.
R² = 0.999Linear from 104 mN to at least 5.8 N. No dead zone, no saturation in range.
2.0% repeatability5.9% hysteresis over 240 presses. Variation falls from 38% to 6% after normalization.
Open sourcePCB design files for every pad size and the readout board are free to download. Off-the-shelf parts, read out at 100 Hz by a Teensy 4.1.

Sensing Principle

Each taxel consists of a photodetector and nearby LEDs under a white translucent silicone layer, capped by a black opaque layer. LED light entering the white translucent silicone is scattered; part of it returns to the detector, and part reaches the black opaque layer and is absorbed. The black opaque layer also shields the detector from ambient light.

Indenting the surface reduces the local thickness and therefore reduces the returned light. A key consequence is that the raw reading decreases with load and depends on the deformation of the elastomer, not on force directly.

In the shear-encoded gel, four-color silicone walls separate the white translucent cubes above the taxels. Shear drags the colored walls sideways, so each detector sees a different mixture of colored light: a detector reports normal deformation through its brightness and shear through its chromaticity.

Drag the sliders. (a) Normal loading produces symmetric deformation, altering the optical paths between the LEDs and the photodetector. (b) Shear loading produces asymmetric deformation, changing the light propagation and intensity received by the photodetector. Interactive version of the paper's figure.

Sensor Design

LuxTac comes in two modes: two interchangeable gel cartridges on the same board and electronics. They share the same materials and base process, and differ in the structure of the white translucent layer.

Shared by both modes

3D CAD render of the 4 by 4 sensing head, photodiodes and LEDs on the flexible PCB
The LuxTac sensing head. TCS34725 RGBC converters and LEDs on the flexible PCB.

Each converter integrates an IR-blocking filter, a photodiode array, and a 16-bit integrating ADC per channel, so the analog signal never leaves the package. All converters share one fixed I²C address, so two TCA9548A switches give each sensor its own bus segment with dedicated 4.7 kΩ pull-ups.

  • Taxels: TCS34725 RGBC light-to-digital converters, pitch 4.42 mm (columns) × 4.80 mm (rows)
  • Illumination: nine 0603 LEDs in a 3×3 grid, each centered between four photodiodes, gated by one NMOS switch
  • Head: 21.8 × 23.8 mm region of a four-layer, 0.2 mm flexible PCB
  • Base: 0.5 mm soft, clear acrylic base plate, part of the stack, screwed to the PCB with four M2 screws
  • Black opaque layer: Psycho Paint Part A, Part B, black Silc Pig, NOVOCS, 1:1:0.5:1 by weight
  • Readout: Teensy 4.1, two TCA9548A I²C switches on two independent buses
  • Frame rate: 100 Hz over USB serial; 2.4 ms integration, 4× gain

Two modes

Mode 1: intensity-encoded gelMode 2: shear-encoded gel
SensesNormal deformationNormal deformation and shear
Channels readClear (C) only, one value per taxelAll four RGBC, four values per taxel
White layerOne continuous white translucent layer, 21.5 × 21.5 mm cavityOne white translucent cube per taxel (about 3.8 × 3.4 mm), 16.4 × 18.0 mm cavity
ColorNoneFour-color silicone walls in 1 mm slits: rows alternate green / red, columns yellow / blue
How it readsPressing thins the layer, so less light returnsBrightness for normal deformation, chromaticity for shear
StackBlack opaque layer, white translucent layer, soft acrylic layer, flexible PCBBlack opaque layer, white translucent layer, four-color silicone walls, soft acrylic layer, flexible PCB
Mold0.5 mm acrylic base, 2 mm acrylic frame, heat-set inserts, M2 screwsSame, plus 3 column walls and 3 row walls with 1.25 mm half-lap notches
Stiffnesskn = 3.86 ± 0.09 N/mm per taxel (normal)1.01 / 1.25 N/mm at 0.5 / 1.5 mm depth (shear)

Mode 1: intensity-encoded gel

Mode 2: shear-encoded gel

Exploded view of the Mode 1 stack: black opaque layer, white translucent layer, soft acrylic layer, flexible PCB
The Mode 1 stack, black opaque layer peeled back.
Exploded view of the Mode 2 stack: black opaque layer, white translucent layer, four-color silicone walls, soft acrylic layer, flexible PCB
The Mode 2 stack, black opaque layer peeled back to show the four-color silicone walls.
The stack peels apart to show the photodiodes and LEDs, then a press dims the light.
A single taxel under normal force, shear, and rotation, then a roller press with the taxel grid lighting up (8×4 board).

One design, many sizes

Every taxel is an independent digital detector on its own bus segment, so the same taxel cell tiles into arrays of other sizes by adding sensors and multiplexers, without changing the readout scheme.

LuxTac pad
Acquisition board

Ordering guide

4×4 board

Drag to rotate · scroll to zoom · right-drag to pan

Loading 3D model…

Flexible PCB exported from KiCad with its copper and silkscreen. Sensor, LED and multiplexer bodies are KiCad library packages of the same size.

Render of the 32×32 LuxTac board under its black silicone skin

Faster Cheaper Better

32×32

1,024 taxels · design

With three-axis force at each taxel.

Contact us for the design files

Coming soon

LuxTac GEN

We are building a platform where you can design your own LuxTac PCB: pick a size and layout, generate the board, and build it yourself.

PCB Generator LuxTac GEN Custom design Contact us

Fabrication

Mode 1: intensity-encoded gel

Mode 2: shear-encoded gel

Characterization

After normalization, every taxel responds linearly to force (R² = 0.999).

Normal force (Mode 1)

Normal-force characterization: normalized signal vs force and depth, force vs depth, and predicted vs measured force Normalized per-taxel uniformity maps and box plot at 0.5, 1.5, 3, and 5.5 N
Normal cartridge, 240 presses: signal vs. force and depth, force from the signal, and uniformity across taxels.

Shear (Mode 2)

Each taxel estimates its own three-axis force, across three indenter shapes and nine held-out sessions.

Shear cartridge: per-taxel and summed three-axis force vs. measured force for three indenters, R-squared per indenter, and the brightness vs. color ablation
Shear cartridge, held-out sessions: per-taxel and summed force vs. measured, R² per indenter, and brightness vs. color.
Characterization summary
MetricValue
Normal cartridge
Noise σ1.06 ± 0.44 counts (0.08% of baseline)
Detection threshold104 ± 49 mN
Working range0.10 to 5.8 N (saturation not reached)
Sensitivity, linearity0.170 FS/N, R² = 0.999
Hysteresis5.9 ± 1.1 %FS
Repeatability2.0 ± 0.3 %FS
Crosstalk, −x / ±y neighbor0.7 ± 1.2 / 5.2 ± 6.0 %
Crosstalk, diagonal neighbor0.8 ± 1.6 %
Per-taxel stiffness kn3.86 ± 0.09 N/mm
Force from h ∘ g−1, RMSE0.20 N (R² = 0.988)
Shear cartridge
Force R², x / y / z, unseen positions0.85 / 0.75 / 0.90
Force R², x / y / z, held-out session0.96 / 0.96 / 0.92
Per-taxel force R², held-out session0.92 / 0.93 / 0.94
Shear direction error, unseen positions (circle) / session14° / 4.8°
Contact location error, unseen positions0.56 mm
Force repeatability between sessions0.05–0.10 N
Normal linearity R²; noise0.996; 0.10% of baseline

Real-time Demonstration

Held-out Presses vs. Ground Truth

Positions never seen in training, replayed on the CNC bench: LuxTac per-taxel forces vs. the ATI force sensor.

Held-out position T01, 3× speed.

Held-out position T06, 3× speed.

Simulation

We simulate LuxTac in IsaacGym with TacSL, on a Franka Panda arm and parallel-jaw gripper, as in the real comparison grasps. Each taxel's penetration depth is turned into the reading the real taxel would give, using the depth curve fitted on the calibration bench, so no raw counts, baselines, or LED drift have to be simulated.

Over 12 grasps of four 3D-printed objects in three orientations, simulated and real contact patterns agree with a mean cosine of 0.942, against 0.59–0.76 for the wrong object, orientation, or taxel order.

A simulated grasp in IsaacGym. Top right: the simulated LuxTac reading.

Sim-to-real alignment (mean cosine)
Real frames compared againstScore
Real frames of the same hold (ceiling)0.9994
Matched simulation0.942
Uniform pad0.763
Simulation of a different object0.674
Same object, different orientation0.642
Matched simulation, taxels shuffled0.594
45° grasps: correct / mirrored diagonal0.946 / 0.264
One-to-one assignment, object recovered10 / 12
Wasserstein-1 distance of pooled values0.067

Robot Experiments

Action Chunking Transformer policies, vision only vs. vision + LuxTac, on the two manipulation tasks.

LuxTac raises peg-in-hole success from 6/15 to 13/15.

Vision + LuxTac clips show the live LuxTac reading in the bottom-left inset,
frame-synced to the camera 0 → 1.

Real-world success rates, with 95% Wilson confidence intervals in brackets
TaskVision onlyVision + LuxTac
Peg-in-hole6/15 (40.0%) [20–64]13/15 (86.7%) [62–96]
Cube sorting5/15 (33.3%) [15–58]6/15 (40.0%) [20–64]
Average36.7%63.3%

BibTeX

@misc{prince2026luxtacflexiblescalableopensource,
      title={LuxTac: A Flexible, Scalable, and Open-Source Optical Taxel Array with Distributed Three-Axis Force Sensing for Tactile Robot Learning}, 
      author={Md Rakibul Islam Prince and John Pelan and Zhiyuan Zhang and Adyaa Khera and Katherine Pesetski and Adeesh Desai and Akimizu Sugiyama and Minwoo Ki and Benjamin Alan Capelle and Temitope Ibrahim Amosa and Zijie Xu and Yihan Huang and Taehun Chung and Long Wang and Wei Li and Shaoting Lin and Yu She},
      year={2026},
      eprint={2611.23575},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2611.23575}, 
}