Event and Relation Extraction for Low-Production and Inefficient Oil Wells Based on LLMs
DOI:
https://doi.org/10.54097/3mewjw81Keywords:
Event Extraction, Large Language Models, Prompt LearningAbstract
To address the challenges of terminology comprehension and insufficient seman-tic relationship modeling in knowledge extraction for petroleum engineering us-ing large LLMs, this paper proposes an Instruction-Heterogeneous Tuning-based Event and Relation Extraction framework (IHT-ERE). The framework employs a triple-heterogeneous instruction strategy, which constructs diversified instruc-tions for each sample and dynamically samples them during training. It integrates LoRA for parameter-efficient fine-tuning, enabling effective adaptation to the oil-field domain with minimal parameter adjustments. During inference, self-consistency decoding and rule-based verification are introduced to enhance output stability. Experimental comparisons based on six mainstream open-source models demonstrate that Mistral-7B fine-tuned with IHT-ERE achieves the best perfor-mance, with a ROUGE-L score of 73.11% and a BERTScore F1 of 92.45%, sig-nificantly outperforming other models under the same framework. Ablation stud-ies confirm that the instruction heterogeneity strategy contributes a core perfor-mance improvement of 3.31 percentage points. This study validates the effective-ness of the IHT-ERE framework combined with LoRA fine-tuning for knowledge extraction in petroleum engineering.
Downloads
References
[1] Hobbs, J. R., Appelt, D., Bear, J., & Israel, D. (1997). FASTUS: A cascaded finite-state transducer for extracting information from natural-language text. MIT Press.
[2] Ahn, D. (2006). The stages of event extraction. In Proceedings of the Workshop on Annotating and Reasoning about Time and Events (pp. 1–8). Association for Computational Linguistics.
[3] Ji, H., & Grishman, R. (2008). Refining event extraction through cross-document inference. In Proceedings of ACL-08: HLT (pp. 254–262). Association for Computational Linguistics.
[4] Nguyen, T. H., Cho, K., & Grishman, R. (2016). Joint event extraction via recurrent neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 300–309). Association for Computational Linguistics.
[5] Chen, Y., Xu, L., Liu, K., Zeng, D., & Zhao, J. (2015). Event extraction via dynamic multi-pooling convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (pp. 167–176). Association for Computational Linguistics.
[6] Choi, H., Kim, J., Joe, S., & Lee, J. (2021). Evaluation of BERT and ALBERT sentence embedding performance on downstream NLP tasks. In 2020 25th International Conference on Pattern Recognition (ICPR) (pp. 5482–5487). IEEE.
[7] Zhu, M., Mao, Y., Cheng, Y., & Wu, J. (2023). Event extraction method based on dual attention mechanism. Journal of Software, 34(7), 3226–3240. (in Chinese)
[8] Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Zhang, Y., Peng, Y., Li, X., Chen, Y., Zhang, H., & Wen, J. R. (2023, November 24). A survey of large language models. arXiv. https://arxiv.org/pdf/2303.18223.pdf
[9] Chen, X., Zhang, N., Xie, X., Deng, S., Tan, C., Huang, F., & Si, L. (2022). KnowPrompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web Conference 2022 (pp. 2778–2788). Association for Computing Machinery.
[10] Lu, Y., Liu, Q., Dai, D., Xiao, X., Lin, H., Han, X., Sun, L., & Wu, H. (2022). Unified structure generation for universal information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 5755–5772). Association for Computational Linguistics.
[11] Zhu, T., Ren, J., Yu, Z., Feng, Y., Zhang, L., Wang, Y., & Wang, R. (2023). MIRROR: A universal framework for various information extraction tasks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 8861–8876). Association for Computational Linguistics.
[12] Jiao, Y., Zhong, M., Li, S., Zhu, H., Wang, P., Zhang, R., Zhang, T., & Liu, Y. (2023). Instruct and extract: Instruction tuning for on-demand information extraction. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (pp. 10030–10051). Association for Computational Linguistics.
[13] Wang, X., Zhou, W., Zu, C., Xia, R., Chen, J., & Zhang, R. (2023, April 17). InstructUIE: Multi-task instruction tuning for unified information extraction. arXiv. https://arxiv.org/pdf/2304.08085.pdf
[14] Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Chen, W., & Chen, W. (2021, October 16). LoRA: Low-rank adaptation of large language models. arXiv. https://arxiv.org/pdf/2106.09685.pdf
[15] Lin, C. Y. (2004). ROUGE: A package for automatic evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out (pp. 74–81). Association for Computational Linguistics.
[16] Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artetxe, M. (2020, February 24). BERTScore: Evaluating text generation with BERT. arXiv. https://arxiv.org/pdf/2002.08835.pdf
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Academic Journal of Applied Sciences
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
