Bringing Near Infrared Spectroscopy To Non-Expert Smartphone Users

The Near Infrared (NIR) is an electromagnetic radiation wave between visible light (Vis) and mid-infrared (MIR), American Society for Testing and Materials (ASTM International) defines the near-infrared spectral region as 780-2526 nm. NIRS is a spectroscopic method that propagates NIR waves through objects and measures the absorbance by diffuse reflection, users could analyse the composition information of objects based on that. NIRS has been applied in a wide range of fields including the pharmaceutical and medical areas. The technology has fast speed and non-destructive analysis with relatively simple requirements for operators, which makes it friendly to non-expert users.

Traditional NIRS scanners used in research laboratory are large and expensive, while recently more and more portable NIRS scanners are appearing with a dramatic drop on price, which makes it much easier for non-expert users to buy and use. Besides, mobile devices are everywhere in people’s life, pairing the technology with mobile devices (smartphones, tablets, etc.) could get rid of other professional operation problems, and bring much more possibilities for non-expert users.

We will explore one such use case, namely smart medication for elderly care. First, we find and review the literature of miniaturized NIRS, exploring whether the impact area would benefit from that. Second, we review literature about wrong medication issues including medication mismanagement and medicine counterfeiting problems, we will do some survey on the elderly taking pills at home to analyse the practicality and feasibility of NIRS technology. Besides, we will conduct the data analysis which trains and predicts based on a set of pharmaceuticals known as knowledge base, with a series of preprocessing techniques and classification algorithms. Finally, we can assess the performance through a user study to evaluate whether the miniaturized NIRS is effective and accurate enough for pharmaceutical identification in non- expert scenarios.

The technology of smartphone is one of the most popular areas in the world, users could purchase and replace smartphones easily as the extremely fast development and update of technology. Because of the convenience with strong utilities, smartphone is becoming an indispensable partner in people’s life.

The technology innovation makes nowadays smartphone a powerful tool, with kinds of integrated sensors which were used in many research fields like healthcare, environment monitor, etc. Most Android smartphones have built-in sensors, that could provide raw data with high precision and accuracy. There are three basic categories of sensors including motion sensors, environmental sensors and position sensors, while more advanced sensors like fingerprint identification sensor, image-sensor and photo-electric sensor are proliferating for end-users. There is a case about smartphones using a photodetector with an accompanying light-emitting diode (LED) light source positioned surrounding the photodetector lens in bioheath area, which proved validation of photoplethysmographic (PPG) acquisition and heart rate (HR) measurement of the new technology. As the trend of technology, it is common that users could self-monitor their health with the help of smartphone with sensors.

Object identification

Object identification is an important technology for finding and identifying objects within a certain environment. The most common method relies on computer vision, mostly making use of an image or a video from a scene, which has remarkably efficiently in detecting many objects by shape and texture, but it is still a challenge for computer vision systems to accurately determine an object because of many factors, one of the most important limitations is not considering objects’ physical composition.

There are 2 cases working on pill detection based on machine vision technology. Hartl et al. (2011) have implemented a smartphone application, and extracted 3 features of the pill containing size, shape and colour. Many computer vision algorithms use grayscale imagery, because the luminance is by far more important in distinguishing visual features than the colour, but the application uses colour as an essential feature for more robust identification, and adopted a series of algorithm based on the work of Susstrunk et al. (2000) to reduce influence of light condition for identification. Cunha et al. (2014) proposed a similar application aimed at elderly medication, whose algorithm was implemented based on four aspects including pill segmentation, shape matching, size estimation and color determination, which paid more attention to computational burden of the algorithm. Both cases are limited to detect external features of pharmaceuticals instead of inner composition, that could be solved by using NIRS effectively.

NIRS with miniaturization

NIRS has ability to penetrate objects up to several millimeters because of its characteristics, thus enabling physical composition analysis which is the advantage of utilizing NIRS rather than computer vision. NIRS has been used in research cross many areas, one of the most classical case is in food analysis. However, most of the previous work has paid much attention on large and heavy NIRS instruments in research fields like laboratories because of the lack of miniaturized NIRS scanners. The miniaturized versions for field use have been developed recently, the equipment now becomes significantly smaller and lighter, which encourages much more research attraction on this area. The NIRS device used in the previous work named DLP NIRscan Nano (Ti.com, n.d.) costs under 1000 US dollars and weighs just 80 grams, which is relatively affordable and portable for end-users. Klakegg et al. (2017) have studied and tested the effect of device motion, sample distance, sample angle, sample surface, sample interference and ambience when using the scanner, and then conclude the proper operations with parameters to overcome the challenges of user-induced error. The result shows that miniaturised NIRS scanners can be successfully used by novice end-users with challenges in consideration. The problem of the method is that all experiments are tested on DLP NIRscan Nano, and some measuring may contain human error as well.

Medication problem

Medicine counterfeiting is a serious problem of the pharmaceutical world. Deisingh (2005) estimated that around 7% of the pharmaceuticals sold in the world are fake. NIRS is a good method to address the challenge of counterfeit drugs. Besides, the previous work by Klakegg et al. (2018) has done a research about medication mismanagement in a nursing center, based on that I would like to talk more about elderly care especially at home.

Nowadays, lots of old people live alone and need to take lots of pills at home, they may be unable to distinguish the pills clearly with complex packages. Cunha et al. (2014) have proposed medication identification application for the elderly, which needs to be refined by decreasing the usage scenario complexity and image noise according to elderly needs. As for our implementation, we want to help them scan the pill easily and get proper feedback giving instructions by using the application. For example, old people might have no good eyesight or literacy to get the words on the pill box, also cannot distinguish between different pills, which brings risk of danger when their children work during the day without time to take care of them. With the help of our application, children could set the app for them in advance, which has an alarm function that could remind the elder to take pill at the proper time (e.g., after lunch). Besides, it can give clear and big signs shown on screen, with loud voice indicating the right or wrong pill they are trying to take.

Conclusion

From the reviewing of above parts, we have introduced the principle and benefit of bringing NIRS into pharmaceutical identification, illustrated the accessibility and necessary of commoditized NIRS towards non-expert users, and inspected previous work about miniaturized NIRS in realistic scenarios including a nursing center. There are still limitations like the lack of machine learning experiments with analysis, the speed of application, the problem of interaction with phone, etc. It would be a great start to expand our work at this point, that we want to provide valid improvements to the previous limitations. Our work flow of the whole system is Phone + NIRS + Server architecture. First, we will scan enough pharmaceuticals for data collection, and then do the preprocessing preparation on the data, after that we can begin the data analysis using multiple machine learning algorithms, analyse and find the best one. The Android application with a 3D printing enclosure could help non-expert users operate and interact with the scanner properly, all complicated analysis is conducted and finished on the server side that users have no need to handle. A final user study would be conducted to evaluate the performance of our project.