Augmentation of NLP neural networks with knowledge graphs

Description

Recently, transformer-based language models, such as famous BERT model (Devlin et al., 2018), have been extremely successful in many downstream natural language understanding (NLU) tasks. Interestingly enough, state-of-the-art models are also able to outperform human baselines on many NLU tasks (https://gluebenchmark.com/leaderboard). However, models like BERT are not quite stable for out-of-domain samples (Park et al., 2019) which makes them harder to use for many real-world applications.

However, humans have the ability to use helpful background knowledge when facing new tasks and try to infuse it with newly learned concepts. This background knowledge can be effectively represented in a structured form like knowledge graph (KG) with fact triplets. This general knowledge background can indeed help us dealing with NLU tasks in new domains, especially when background information is crucial like question answering. There are some previous efforts that try to augment traditional neural networks with KG (Annervaz et al., 2018) in a task-agnostic manner and yield considerable boost over without-KG baseline.

In this project, the candidate will exploit the recent advancements in KG representation and also different knowledge extraction algorithms that could be used together with a conventional deep NLP model for different NLU tasks.

References

• Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
• Park, C., Kim, J., Lee, H.G., Amplayo, R.K., Kim, H., Seo, J. and Lee, C., 2019. ThisIsCompetition at SemEval-2019 Task 9: BERT is unstable for out-of-domain samples. arXiv preprint arXiv:1904.03339.
• Annervaz, K.M., Chowdhury, S.B.R. and Dukkipati, A., 2018. Learning beyond datasets: Knowledge graph augmented neural networks for natural language processing. arXiv preprint arXiv:1802.05930.
  

Prerequisites

The candidate should have programming experience, ideally in Python. Previous experience with machine learning and natural language processing is a plus.

30% Theory, 30% Implementation, 40% Research and Experiments

Contact

Send me your CV: [email protected].