Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/CW64301.2024.00028

Other Link： https://www.computer.org/csdl/proceedings-article/cw/2024/271700a195/259PFR330Eo

Language：English   Publishing type：Research paper (scientific journal)  

DOI： 10.1109/ACCESS.2024.3434742

researchmap

Language：Japanese   Publishing type：(MISC) Lecture materials etc. (seminar, tutorial, course, lecture and others)  

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

DOI： 10.1109/APSIPAASC63619.2025.10849072

Authorship：Lead author   Language：English   Publishing type：Research paper (scientific journal)  

Text detection is a crucial pre-processing step in optical character recognition (OCR) for the accurate recognition of text, including both fonts and handwritten characters, in documents. While current deep learning-based text detection tools can detect text regions with high accuracy, they often treat multiple lines of text as a single region. To perform line-based character recognition, it is necessary to divide the text into individual lines, which requires a line detection technique. This paper focuses on the development of a new approach to single-line detection in OCR that is based on the existing Character Region Awareness For Text detection (CRAFT) model and incorporates a deep neural network specialized in line segmentation. However, this new method may still detect multiple lines as a single text region when multi-line text with narrow spacing is present. To address this, we also introduce a post-processing algorithm to detect single text regions using the output of the single-line segmentation. Our proposed method successfully detects single lines, even in multi-line text with narrow line spacing, and hence improves the accuracy of OCR.

DOI： 10.1587/transinf.2023EDP7070

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

シャインマスカットという品種のブドウにおいて，実際にブドウを収穫してよいかどうかの判断の一つとして色が使われる。山梨県果樹試験場が開発したシャインマスカットブドウ用のカラーチャートでは5段階の色の変化が示されている。本稿では，実際のブドウの写真からカラーチャートに基づくブドウの色を推定する深層学習モデルを検討した。提案モデルでは，RGB以外の色空間としてLAB空間を採用し，RGBのGBチャンネルとLABのLチャンネルを組み合わせることで，色推定の精度が改善することが分かった。

CiNii Books

CiNii Research

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

This study explores the application of metric learning in an end-to-end multilingual automatic speech recognition (ASR) model, employing the wav2vec 2.0 framework. In the proposed method, the E2E ASR model implements metric learning by obtaining acoustic features corresponding to character labels through forced alignment. When metric learning was applied to a six-language E2E ASR model during training, the model incorporating metric learning demonstrated a 0.7-point improvement in the character error rate (from 8.4% to 7.7%) over the baseline model, which was trained without metric learning. Additionally, the visualization of feature vectors indicated a decrease in both the variation of acoustic feature vectors for individual characters and inter-character interference, further underscoring the effectiveness of our approach.

DOI： 10.1109/GCCE59613.2023.10315608

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

Fisheye cameras present a challenge for face recognition due to their wide-angle perspective and image distortion. This paper introduces a novel approach to enhancing training data for fisheye-based face recognition without requiring image calibration. We employ five image-remapping transformations to diversify and expand the training dataset and evaluate the effectiveness of this approach using deep learning networks: HRNetV2 and ResNet50. The results demonstrate significant improvements in classification accuracy when utilizing authentic fisheye facial data. Specifically, HRNetV2 exhibits an increase of 30.2%, while ResNet50's performance improves by 11.8% compared to their respective baseline performances. This study presents a fresh method for refining face recognition in fisheye camera scenarios, thereby extending its potential for real-world applications.

DOI： 10.1109/GCCE59613.2023.10315437

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)   Publisher：The Japan Society of Mechanical Engineers  

Our group is researching a support system for cultivating Shine-Muscat grapes. The ripeness of the grapes is determined by their color and is compared against a Shine-Muscat color chart. In this paper, we introduce a color estimation model that can determine the optimal time for harvesting by using a deep learning model to estimate the color of the grape bunch. We incorporate metric learning during model training and use the specific color space representation of images to achieve high accuracy in our color estimation model. In addition, we propose a method to incorporate an appropriate reference color into the color estimation model. Our evaluation experiment indicatesd that our proposed model correctly identified the optimal time to harvest for 85.9% of the grapes.

DOI： 10.1299/jsmermd.2023.2a1-b03

CiNii Research

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

This paper examines the potential of deep learning methods for grape quality evaluation, particularly determining ripeness suitable for harvest and estimating grape 'sweetness,' using imagery. The uniqueness of this study lies in its exclusive use of images captured with standard cameras to predict the quality of an edible green grape variety known as 'Shine Muscat.' This approach anticipates future scenarios where farmers can easily capture images using smart glasses or smartphones. The method we developed leverages three channels from the RGB color space and CIELAB color space of the grape image, utilizes a feature extractor employing a vision transformer-based model, and introduces a deep learning model training method using a reference color. These innovations have successfully elevated the accuracy of the harvest determination task to 87.7%. Additionally, though the accuracy is not yet high, the paper showcases the potential of estimating grape 'sweetness' directly from images by using the color estimation model as a pre-training model.

DOI： 10.1109/MetroAgriFor58484.2023.10424087

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

This paper presents a novel data augmentation technique crucial for training AI-OCR systems for handwritten character classification. Using a Y-autoencoder (Y-AE) enhanced with Adaptive Instance Normalization, diverse handwriting styles are generated to improve the breadth of handwriting representations. A filtering mechanism is introduced to include only valid character images for training. The method was tested on a subset of the ETL Character Database, featuring 92 unique Japanese Hiragana and Katakana characters. The baseline classifier achieved an accuracy of 0.9061. However, when using the augmented dataset, which included Y-AE model-generated and filtered images, the accuracy improved to a maximum 0.9555 with data augmentation technique. These results showcase the potential of this data augmentation technique in consumer electronics, particularly in AI-OCR software. Despite needing some noise removal, the approach significantly boosts classifier accuracy, suggesting an efficient way forward for document processing in various sectors.

DOI： 10.1109/GCCE59613.2023.10315447

Scopus

researchmap

Language：English   Publishing type：Research paper (scientific journal)  

The color of a bunch of grapes is a very important factor when determining the appropriate time for harvesting. However, judging whether the color of the bunch is appropriate for harvesting requires experience and the result can vary by individuals. In this paper, we describe a system to support grape harvesting based on color estimation using deep learning. To estimate the color of a bunch of grapes, bunch detection, grain detection, removal of pest grains, and color estimation are required, for which deep learning-based approaches are adopted. In this study, YOLOv5, an object detection model that considers both accuracy and processing speed, is adopted for bunch detection and grain detection. For the detection of diseased grains, an autoencoder-based anomaly detection model is also employed. Since color is strongly affected by brightness, a color estimation model that is less affected by this factor is required. Accordingly, we propose multitask learning that uses metric learning. The color estimation model in this study is based on AlexNet. Metric learning was applied to train this model. Brightness is an important factor affecting the perception of color. In a practical experiment using actual grapes, we empirically selected the best three image channels from RGB and CIELAB (L*a*b*) color spaces and we found that the color estimation accuracy of the proposed multi-task model, the combination with “L” channel from L*a*b color space and “GB” from RGB color space for the grape image (represented as “LGB” color space), was 72.1%, compared to 21.1% for the model which used the normal RGB image. In addition, it was found that the proposed system was able to determine the suitability of grapes for harvesting with an accuracy of 81.6%, demonstrating the effectiveness of the proposed system.

DOI： 10.1007/s00371-022-02666-0

Scopus

researchmap

Language：English   Publishing type：(MISC) Institution technical report and pre-print, etc.  

This paper proposes a model for transforming speech features using the
frequency-directional attention model for End-to-End (E2E) automatic speech
recognition. The idea is based on the hypothesis that in the phoneme system of
each language, the characteristics of the frequency bands of speech when
uttering them are different. By transforming the input Mel filter bank features
with an attention model that characterizes the frequency direction, a feature
transformation suitable for ASR in each language can be expected. This paper
introduces a Transformer-encoder as a frequency-directional attention model. We
evaluated the proposed method on a multilingual E2E ASR system for six
different languages and found that the proposed method could achieve, on
average, 5.3 points higher accuracy than the ASR model for each language by
introducing the frequency-directional attention mechanism. Furthermore,
visualization of the attention weights based on the proposed method suggested
that it is possible to transform acoustic features considering the frequency
characteristics of each language.

arXiv

researchmap

Other Link： http://arxiv.org/pdf/2203.15473v1

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

深層学習を用いた文字認識システムでは，高性能な文字認識器を学習させるために，大量のデータが必要であることはよく知られている。本稿では，Y-Autoencoder （Y-AE） ベースの手書き文字生成器を導入し，1枚の画像から複数の日本語文字を生成することで，手書き文字認識器の学習用データ量を増やす。実験の結果，Y-AEは日本語の文字画像を生成でき，手書き文字認識器の学習に用いることができ，手書き文字認識精度（F1スコア）を向上することがわかった。

CiNii Books

CiNii Research

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

It is well-known that the deep learning-based optical character recognition (OCR) system needs a large amount of data to train a high-performance character recognizer. However, it is costly to collect a large amount of realistic handwritten characters. This paper introduces a Y-Autoencoder (Y-AE)-based handwritten character generator to generate multiple Japanese Hiragana characters with a single image to increase the amount of data for training a handwritten character recognizer. The adaptive instance normalization (AdaIN) layer allows the generator to be trained and generate handwritten character images without paired-character image labels. The experiment showed that the Y-AE could generate Japanese character images then used to train the handwritten character recognizer, producing an F1-score improved from 0.8664 to 0.9281. We further analyzed the usefulness of the Y-AE-based generator with shape images and out-of-character (OOC) images, which have different character image styles in model training. The result showed that the generator could generate a handwritten image with a similar style to that of the input character.

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper proposes a method to improve the performance of multilingual automatic speech recognition (ASR) systems through language- and speaker-independent feature transformation in a framework of end-to-end (E2E) ASR. Specifically, we propose a multi-task training method that combines a language recognizer and a speaker recognizer with an E2E ASR system based on connectionist temporal classification (CTC) loss functions. We introduce the language and speaker recognition subtasks into the E2E ASR network and introduce a gradient reversal layer (GRL) for each sub-task to achieve language and speaker-independent feature transformation. The evaluation results of the proposed method in the multilingual ASR system in six sorts of languages show that the proposed method achieves higher accuracy than the ASR models for each language by introducing multi-tasking and GRL.

DOI： 10.21437/Interspeech.2021-390

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

The color of grape bunches is a significant factor when harvesting grapes at the appropriate timing. Judging the suitable color for shipment requires experience and varies from one person to another. We herein describe a support system for grape harvesting based on color estimation. To estimate the color of a bunch of grapes, bunch detection, grain detection, removal of diseased grains, and color estimation should be performed. Models based on deep learning are employed for this series of processes. Since color is strongly affected by sunlight, we propose a multitask model that considers sunlight exposure to achieve a robust color estimation model that exhibits decreased sensitivity to sunlight. Our results show that the color estimation accuracy of the model is 76% when sunlight exposure is not considered and 81% when sunlight exposure is considered. In addition, we performed a practical field test of the developed harvest support system in an actual grape field. The results show that our support system can determine the appropriateness of grape harvest with an accuracy of 90%, demonstrating the effectiveness of the system.

DOI： 10.1109/CW52790.2021.00040

Scopus

researchmap

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

When applying voice activity detection (VAD) to a noisy sound, in general, noise reduction (speech separation) and VAD are performed separately. In this case, the noise reduction may suppress the speech, and the VAD may not work well for the speech after the noise reduction. This study proposes a VAD model through the tandem connection of neural network-based noise separation and a VAD model. By training the two models simultaneously, the noise separation model is expected to be trained to consider the VAD results, and thus effective noise separation can be achieved. Moreover, the improved speech/noise separation model will improve the accuracy of the VAD model. In this research, we deal with real-live speeches from baseball games, which have a very poor signal-to-noise ratio. The VAD experiments showed that the VAD performance at the frame level achieved 4.2 points improvement in F1-score by tandemly connecting the speech/noise separation model and the VAD model.

DOI： 10.21437/Interspeech.2021-792

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

This paper describes the ExKaldi-RT online automatic speech recognition (ASR) toolkit that is implemented based on the Kaldi ASR toolkit and Python language. ExKaldi-RT provides tools for building online recognition pipelines. While similar tools are available built on Kaldi, a key feature of ExKaldi-RT that it works on Python, which has an easy-to-use interface that allows online ASR system developers to develop original research, such as by applying neural network-based signal processing and by decoding model trained with deep learning frameworks. We performed benchmark experiments on the minimum LibriSpeech corpus, and it showed that ExKaldi-RT could achieve competitive ASR performance in real-time recognition.

DOI： 10.1109/GCCE53005.2021.9621992

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

Inflorescence trimming is a crucial process to produce high-quality table grapes. It can eliminate nutrient competition in a bunch and makes it less vulnerable to disease development. After trimming, the remaining part of the inflorescence should have a target length decided by the grape variety. This is challenging for novice farmers because of the time constraint. The farmer needs to finish trimming the inflorescence before the berries develop. This paper proposes a novel end-To-end inflorescence length measurement method for supporting a trimming process with augmented reality technology. The proposed technique makes use of the state-of-The-Art deep neural network model for detecting the inflorescence area, as well as the scissors from the images captured with a camera installed on an optical see-Through head-mounted display. A new algorithm is designed to estimate the length of the remaining inflorescence with the screw of the scissors loop as the calibrator. The estimated length is then visualized on the head-mounted display to support the farmer in performing the trimming correctly and efficiently. The experiment, conducted with real inflorescence trimming tasks, shows that the mean absolute error of the length estimation is only 0.19 cm, which is small enough for use in real applications.

DOI： 10.1109/CW52790.2021.00022

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

In the hospitality industry, including convenience stores and airport service counters, operational staff must be trained to serve customers satisfactorily and to avoid problems with them. This study investigates a spoken dialog training system for improving customer service by operational staff. In a conventional spoken dialog system, a system user uses the dialog system as a customer, and the dialog agent assists the system user in fulfilling his or her requirements. In our system, in contrast, the dialog agent plays the role of the customer. Consequently, the behaviors of the human and the customer are opposite to those in a traditional dialog system, and there has thus far been no research on such systems. This paper introduces a prototype of such a system that we have developed and describes a simple experiment. The results of the experiment confirm the usefulness of our system for hospitality training.

Scopus

researchmap

Language：English   Publishing type：Research paper (international conference proceedings)  

We present ExKaldi, an automatic speech recognition (ASR) toolkit, which is implemented based on the Kaldi toolkit and Python language. While similar Kaldi wrappers are available, a key feature of ExKaldi is an integrated strategy to build ASR systems, including processing feature and alignment, training an acoustic model, training, querying N-grams language model, decoding and scoring. Primarily, ExKaldi builds a bridge between Kaldi and deep learning frameworks to help users customize a hybrid hidden Markov model-deep neural network-based ASR system. We performed benchmark experiments on the TIMIT corpus and revealed that ExKaldi could build a system from scratch with Python and achieved reasonable recognition accuracy. The toolkit is open-source and released under the Apache license.

DOI： 10.1109/GCCE50665.2020.9291717

Scopus

researchmap

Authorship：Lead author   Language：English   Publishing type：Research paper (international conference proceedings)  

In this study, a real-time automatic speech recognition (ASR) system based on the Kaldi ASR toolkit, with low-latency and customizable models, without any internet connection, was developed. The proposed ASR system includes a voice activity detection (VAD) module and an audio transmitter as a front-end speech processing and a decoder for the received audio signals. The ASR system was evaluated in terms of ASR accuracy and speech processing speed. Consequently, the ASR system achieved high ASR accuracy on the CSJ (Corpus of Spontaneous Japanese) test set with super low-latency.

DOI： 10.1109/GCCE50665.2020.9291818

Scopus

researchmap

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)   Publisher：[東京] : Institute of Electrical Engineers of Japan  

CiNii Research

researchmap

Other Link： http://id.ndl.go.jp/bib/031476283

Language：English   Publishing type：Research paper (international conference proceedings)  

In outdoor environments, fluctuating airflow and gas distribution make gas source localization (GSL) tasks difficult. In our research, we use neural networks (NNs) to overcome these difficulties by applying long short-Term memory deep neural networks (LSTM-DNNs) to time series data taken from a gas sensor array and anemometer to estimate the position of a gas source. In this paper, we present NNs for GSL with the ability to use various length input data and estimate a gas source location each time-step. In doing so, we were able to estimate the location of a gas source within 40 time-steps (20 s) and achieved (using 300 time-steps) an estimation accuracy of 95%.

DOI： 10.1109/ISOEN.2019.8823160

Scopus

researchmap

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Authorship：Lead author   Language：Japanese   Publishing type：Research paper (research society, symposium materials, etc.)  

J-GLOBAL

researchmap

Language：Japanese   Publishing type：Research paper (scientific journal)  

Recently, deep learning technologies have been in the spotlight. Deep learning is one of a powerful technology to classify or recognize objects which captured by a camera. Such application has a high affinity with Internet-of-Things (IoT) devices. Therefore, it is considered that these technologies are used in embedded systems and IoT devices. In this paper, we verify deep learning applications like image classification can work well on a small computer such as Raspberry Pi. We develop three deep learning applications by using two types of deep learning frameworks (libraries). We prepare four types of small computers, and these applications are tested on the computers. In addition, we also investigate the relationship between the processing time, the memory consumption and the number of parameters of the deep neural network model. The verification experiments show that a program based on a deep learning library implemented by C++ language fast run and simple neural network models could work in real-time on small computers. Besides, the other experiment also clears that the more parameters increase the processing time and the consumption memory in proportion without depending on the deep learning libraries and small computers.

DOI： 10.1541/ieejeiss.138.1108

Scopus

researchmap

Event date： 2024.5

Language：Japanese   Presentation type：Oral presentation(general)  

Venue：浜松市   Country：Japan