A smart livestock framework for multimodal abnormally screaming pig identification and behavioral reporting in group-housed environments

초록

Pigs produce abnormal vocalizations, such as screams, in response to stress, leading to reduced growth rates, increased infection risk, and productivity declines. An intelligent monitoring system is essential for detecting these responses and enabling rapid intervention, particularly in group-housed environments. This study proposes a multimodal framework integrating audio and video modalities to identify screaming pigs and analyze their behaviors. The framework comprises four components: (1) audio-video input preprocessing, (2) context-based feature construction, (3) context refinement and identification, and (4) large language model (LLM)-based behavioral report generation. Raw data are collected from CCTV and microphones, followed by noise reduction and Mel-spectrogram conversion. A CNN-based encoder extracts visual and auditory features, which are fused to construct multimodal feature maps. Self-attention and an LSTM network then identify the screaming pig by integrating temporal and contextual information. Finally, an LLM generates behavioral analysis reports. Experiments using real-world data from a commercial pig farm demonstrated the system's effectiveness, achieving a mean average precision (mAP) of 0.747. The study also compared LLM-generated reports from GPT-4o, DeepSeek-R1, and Llama-3.1-8B. This is the first study to integrate multimodal data for identifying screaming pigs and generating behavioral reports, validating its potential as a foundational technology for smart livestock management and animal welfare improvement.

키워드