不明觉厉
Chinese team develops intelligent vision sensor that turns light signals into AI-ready tokens, reducing use of energy_我的网站
一 |
A Chinese research team from Nanjing University has developed a new ultra-low-power intelligent vision sensor chip, dubbed "LightTok," that can convert light signals into tokens within the sensor, significantly reducing the high energy consumption caused by frequent transfers of massive amounts of redundant data, the principal investigator told the Global Times.
According to a release from the Institute of Brain-Inspired Intelligence of Nanjing University, tokens generated by the LightTok chip can be directly fed into a Transformer encoder for image recognition.
"Our design idea was to move token generation onto the sensor itself, allowing the chip to directly produce tokens that AI models can process once light reaches the sensor," Miao Feng, director of the Institute of Brain-Inspired Intelligence at Nanjing University, told the Global Times on Thursday. "These tokens contain complete image information."
Physical AI refers to intelligent systems capable of autonomously perceiving, reasoning, acting and receiving feedback in the real world, representing a key pathway for AI to move from the digital realm into the physical world. Vision-based physical AI systems powered by large AI models need to convert visual information from real-world environments into tokens that can be processed by AI models before feeding these tokens into Transformers for subsequent tasks.
In traditional visual perception pipelines, light signals must go through multiple stages, including image sensing, analog-to-digital conversion, data buffering and transfer, digital image patching and embedding, before being transformed into tokens that AI models can process. The frequent transfer of massive amounts of redundant data has resulted in high energy consumption at the edge, according to a report by Science and Technology Daily.
The LightTok chip directly addresses a key challenge in physical AI hardware: how to efficiently acquire and tokenize visual information from the physical world with low energy consumption, Miao said.
The LightTok chip consists of a photosensitive memory array and peripheral circuits. The team built the array based on single-layer molybdenum disulfide (MoS₂) floating-gate phototransistors, with each pixel capable of sensing light, storing information and performing analog computing. By processing optical information directly within the chip, the device can convert captured visual signals into tokens for AI models, according to the research team.
The current LightTok prototype has a resolution of 32×32 pixels, or 1,024 photosensitive pixels, which is still smaller than that of smartphone cameras and industrial imaging systems. However, Miao said the technology is compatible with CMOS manufacturing processes and can be scaled up. With wafer-level growth of molybdenum disulfide materials, the chip could potentially achieve a scale comparable to existing imaging devices.
Miao said the chip also draws inspiration from the information-processing mechanism of human vision. Similar to how the retina extracts key visual information before transmitting it to the brain, the chip also aims to process visual information at an early stage.
The research was conducted in collaboration with another research team from the National University of Singapore. The findings were published on Wednesday in Nature Sensors, an internationally renowned journal in the field of sensing technology, according to the release.
Potential applications for LightTok include drone systems, autonomous remote sensing and small-scale embodied AI systems, Miao said. In these scenarios, devices need to continuously detect, understand and track targets, generating massive amounts of visual data. By reducing the energy required for visual processing, the technology could extend the operating time of drones, satellites and small robots with limited power supplies, the expert said.
。 本篇评级为★★★，主要围绕以下内容展开： 1、战略资源品赛道，需要注意的年尾行情时间节点 2、科技赛道"冒泡"，机会大吗？ 如有疑问请以音频内容为准，添加妙投小虎哥微信miaotou515，入群有机会与董小姐进一步交流。 新闻解读评级说明：五星重磅，四星重要，三星级以下大家选择听。 （文稿为机器转录，如有出入，请以音频为准） 每天10分钟拆解新闻背后的逻辑。
二 | 大家好，今天是2015年的11月17号星期一，我是董小姐。 1、战略资源品赛道，需要注意的年尾行情时间节点 今天国内三大指数微跌，内部分化却相当明显：涨的酣畅淋漓，跌的也毫不含糊。 先看上涨阵营——受“涨价”驱动，上游能源金属及连带化工板块继续高光。碳酸锂期货主力合约日内飙涨逾9%，直接点燃战略资源品赛道热情；电池板块整体涨幅突破1%，与电池挂钩的精细化工亦乘势再上。
三 | 不过，狂欢之余仍需在心里划一条“时间红线”：年底翻篇进入2026年前，行情可能维持热闹——储能项目抢工、新能源车冲刺订单（明年购置税将上调，需求前置），但能源品、电池核心原材料会否提前透支需求？投资者心里要有谱。
四 | 本文为妙投付费内容，上述仅为摘要，购买本专栏即可解锁完整内容。新用户可免费领本专栏7天阅读体验机会，在妙投APP-我的-权益兑换 输入“妙投888”即可领取。
五 |
Current article:http://i7qtvo8.ruidunguizanpaorangchongouxia.sbs/list_re6e4t/w5k.html
Published on:00:00:00