Monday, 19 January 2015

Mixing Text Mining with Semantic Technologies - sample application.

The very broad subject of processing the natural language is incredibly hot nowadays. In many cases, a regular text mining approach is not adequate to the problems that we are facing. Therefore text mining methods are mixed with Natural Language Processing(NLP) methods, like also, with semantic technologies - what gives better results. One of such a problem is how to find out, if two sentences are semantically equal or not.

The solution for the above problem could be used on many fields. One of them is detection of an abusive clauses inside a contract. Sometimes it's really hard to understand correctly, the exact meaning of a clause inside a contract, even for a specialists. For a sake of presentation I have developed a simple application prototype which attempts to solve this problem. Application was developed in C# and it uses Ontorion SDK.

Input

Before running the application we need three files:
  1. File with contract in which we will attempt to detect abusive clauses.
  2. File with abusive clauses.
  3. File with ontology.

Files 1&2 can be in Ms Word or txt format:


Figure 1


The ontology file was created in Fluent Editor. It contains knowledge extracted from the files 1&2. In this sample the ontology is very small and simple: 



Figure 2

Above ontology was exported to the Ontorion so I could easily explore it from within application, using Ontorion SDK.

In application we just load file with contract and file with abusive clauses, next specify the name of the Ontorion database in which the ontology resides and then we can start the matching process.

Figure 3 

Algorithm

An algorithm of matching sentences does the following:
  1. Sentences detection of files 1&2.
  2. Tokenization of files 1&2 sentences.
  3. Stemming of each sentence words in files 1&2.
  4. Obtaining the knowledge from ontology by using Ontorion SDK - the relations between concepts/instances along with its annotations.
  5. Calculating the improved semantic cosine similarity measure between all of file 1 and file 2 sentences.
This algorithm doesn't take in account a words order, what in this particular case will be an advantage. The most important part of whole process plays an ontology. More "rich" the ontology is, more accurate results we will get.

Example

As an example I have prepared one contract clause and one abusive clause. Ontology used is same as on Figure 2. We will start from the end, so from the result of comparison, and then describe it:

Figure 4

So the contract clause is as follow:

Payment for the training: after the charge, money is not refundable.

and an abusive clause is:

Payment for the service: after the payment the money is not returnable.

From the relations described in ontology we know that:


Every training is a service.

thus, "training" and "service" words, can be considered here as semantically equal. In our ontology we also defined an annotations for each instance/concept, which are considered as synonyms. So for "payment" concept we defined, inter alia, a "charge" annotation, and for "refundable" concept - "returnable" annotation. So despite the differences between words used in both sentences, the meaning of them is same, that's why the semantic similarity rate equals 1.

You can see this sample application in action on below video:






To learn more about Ontorion visit: http://www.cognitum.eu/semantics/


342 comments:

  1. The expansion of internet and other business intelligence leads to large volume of data. Industries are looking for talented professionals to maintain and process huge volume of data with latest tools available in the market. Taking Hadoop Training in Chennai | Big Data Training in Chennai will ensure better career prospects for talented professionals.

    ReplyDelete
    Replies
    1. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. machine learning projects for final year In case you will succeed, you have to begin building machine learning projects in the near future.

      Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.


      Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.

      Delete

  2. This was an nice and amazing and the given contents were very useful and the precision has given here is good.
    Data Science Training in Chennai

    ReplyDelete
  3. good one,very onformative thanks for sharing your views and ideas.. it is very useful to me.. thanks once again
    Python Training in Chennai

    ReplyDelete
  4. Thank you.. your blog is doing good work. Awesome blog. Devops Training in Chennai

    ReplyDelete
  5. The concept of DevOps is founded on building a culture of collaboration between teams that historically functioned in relative siloes. The promised benefits include increased trust, faster software releases, ability to solve critical issues quickly, and better manage unplanned work.Devops Training in Chennai
    Devops Training in Bangalore

    ReplyDelete
  6. Wow, Nice blog. Thank you so much for the efforts of this blog. Visit for
    Best Offshore VPS

    ReplyDelete
  7. Thanks for your interesting ideas.the information's in this blog is very much useful for me to improve my knowledge.
    app development course in bangalore
    Android Training in Perambur
    Android Training in Nungambakkam
    Android Training in Karapakkam

    ReplyDelete
  8. Thanks for the information, worth reading article.

    IoT Training Pune

    ReplyDelete
  9. Played on BGAOC with big wins? NOT? Come to us as soon as possible and win with us. perfect all slots casino Come get your chance to win.

    ReplyDelete
  10. It has been just unfathomably liberal with you to give straightforwardly what precisely numerous people would've promoted for an eBook to wind up making some money for their end, basically given that you could have attempted it in the occasion you needed.
    iosh safety course in chennai

    ReplyDelete
  11. Congratulations guys, quality information you have given!!!
    Regards,
    Data Science Course In Chennai

    ReplyDelete
  12. Thanks for giving great kind of information. So useful and practical for me. Thanks for your excellent blog, nice work keep it up thanks for sharing the knowledge.
    dining room interior designer in noida

    ReplyDelete
  13. nice work keep it up thanks for sharing the knowledge.Thanks for sharing this type of information, it is so useful.
    Epoxy Grout manufacturer

    ReplyDelete
  14. Very Clear Explanation. Thank you to share this
    Regards,
    Regards,
    Best Devops Training Institute in Chennai

    ReplyDelete
  15. Супер отличная гибкая світлодіодна стрічка на любой вкус и цвет, обычно покупаю в интернет магазине.

    ReplyDelete
  16. This is best one article so far I have read online, I would like to appreciate you for making it very simple and easy
    Regards,
    Devops Training in Chennai | Devops Certification in Chennai

    ReplyDelete
  17. You are doing a great job. I would like to appreciate your work for good accuracy.
    devops certification in chennai

    ReplyDelete
  18. Amazing! I like to share it with all my friends and hope they will like this information.
    Regards,
    Python Training in Chennai | Python Programming Classes | Python Classes in Chennai

    ReplyDelete
  19. Thanks for sharing this information admin, it helps me to learn new things. Continue sharing more like this.
    Regards,
    Tableau training in Chennai | Tableau Courses Training in Chennai | Tableau training Institute in Chennai

    ReplyDelete
  20. I feel happy about and learning more about this topic. keep sharing your information regularly for my future reference. This content creates a new hope and inspiration with in me. Thanks for sharing article like this. the information which you have provided is better then other blog.
    IELTS Coaching in Dwarka

    ReplyDelete
  21. This comment has been removed by the author.

    ReplyDelete
  22. Thank you for this great article i learn a lot from your article keep it up.
    attitude status in hindi
    Life status in hindi
    Love Status in hindi

    ReplyDelete
  23. many peoples want to join random whatsapp groups . as per your demand we are ready to serve you whatsapp group links . On this website you can join unlimited groups . click and get unlimited whatsapp group links

    ReplyDelete
  24. Interesting information and attractive.This blog is really rocking... Yes, the post is very interesting and I really like it.I never seen articles like this. I meant it's so knowledgeable, informative, and good looking site. I appreciate your hard work. Good job.
    Kindly visit us @
    Sathya Online Shopping
    Online AC Price | Air Conditioner Online | AC Offers Online | AC Online Shopping
    Inverter AC | Best Inverter AC | Inverter Split AC
    Buy Split AC Online | Best Split AC | Split AC Online
    LED TV Sale | Buy LED TV Online | Smart LED TV | LED TV Price
    Laptop Price | Laptops for Sale | Buy Laptop | Buy Laptop Online
    Full HD TV Price | LED HD TV Price
    Buy Ultra HD TV | Buy Ultra HD TV Online
    Buy Mobile Online | Buy Smartphone Online in India

    ReplyDelete
  25. The article is very interesting and very understood to be read, may be useful for the people. I wanted to thank you for this great read!! I definitely enjoyed every little bit of it. I have to bookmarked to check out new stuff on your post. Thanks for sharing the information keep updating, looking forward for more posts..
    Ruby

    ReplyDelete
  26. I like your post very much. It is very much useful for my research. I hope you to share more info about this. Keep posting!!
    When operating system no longer can put data on RAM then it uses some area of hard disk space to store data that area is called Swap space. When the system goes on low memory mode then it uses Swap space to store data. Basically, it swaps the data on RAM of an ideal program to swap area. In this tutorial, we are going to learn how to add swap space on Linux Mint 19.

    Prerequisites
    Before you start to create sudo user on Linux Mint 19. You must have the root user account credentials of your system.
    Read More

    ReplyDelete
  27. Amazing article. Your blog helped me to improve myself in many ways thanks for sharing this kind of wonderful informative blogs in live.
    mobile App Development Training in Chennai | Android Development Training in Chennai | Ios App Development Training in Chennai

    ReplyDelete
  28. Thanks for posting this highly informative article. I feel glad about learning more about this concept. Maintain the number of posting and keep up the good work!
    Data Science Course in Chennai | Data Science Training in Chennai

    ReplyDelete
  29. Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man,Keep it up.
    New Zealand education Consultants in Delhi

    ReplyDelete
  30. Thanks for the Valuable information.Really useful information. Thank you so much for sharing.It will help everyone.Keep Post. Find Some Indian Memes. Interesting News Tamilrockers Movie Download Trending News Some Life hacks tips Life hacks Entertainment News Find Some Viral News Here.Trending News
    saurabh jindal

    ReplyDelete
  31. How To Download PUBG Mobile Lite iOS Free – For Android, IOS and PC – Tips and APK Download

    ReplyDelete
  32. Thanks for sharing valuable information.It will help everyone.keep Post.
    Kerala Lottery Guessing

    ReplyDelete
  33. Nice article
    Thanks for sharing the information
    Please visit leadmirro to know your blog rank

    ReplyDelete
  34. amazing post written ... It shows your effort and dedication. Thanks for share such a nice post.
    sandeep maheshwari quotes and harry potter wifi names

    ReplyDelete
  35. Flox Blog Beginner's Guide for Web Developers.


    Nice article. It's very helpful to me. Thank you. Please check my online rgba color picker tool.


    Check your website is mobile friendly or not with Responsive Checker.


    Convert your text to Sentence Case. Then go for Flox Sentence Case Converter.

    ReplyDelete
  36. Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man, Keep it up.
    Veteran Mode, MLive Mod APK, Layon Shop, Multitas Pinjaman, Brasil Tv New, Project IGI, Enlight Pixaloop Pro, Gimy TV, Sakura Live Show China, TR Vibes HotStar

    ReplyDelete
  37. I discovered your blog while scanning for the updates, I am glad to be here. Exceptionally helpful substance and furthermore effectively reasonable giving.. Trust me I did composed a post about instructional exercises for amateurs with reference of your blog.fruits drawing

    ReplyDelete
  38. Thank you for sharing the valuable information. Thanks for providing a great informatic blog, really nice required information & the things I never imagined. Thanks you once again Download Ludo King Mod Apk

    ReplyDelete
  39. Thank you for sharing such valuable information and tips. This can give insights and inspirations for us; very helpful and informative! Would love to see more updates from you in the future.
    CLICK HERE

    ReplyDelete
  40. Thank you for this great article i learn a lot from your article keep it up.
    How to Install Yarn on Ubuntu 18.04

    ReplyDelete
  41. Today i found a new sexy bike Suzuki Gixxer SF 250 , its look so beautiful is available in India at a price of Rs. 1.71 - 1.71 Lakh ex-showroom

    ReplyDelete
  42. KTM Bike launch a super bike KTM 790 Duke in India at a price of Rs. 8.64 Lakh ex-showroom Delhi, Check it MotoBike

    ReplyDelete
  43. Wow What A Nice And Great Article, Thank You So Much for Giving Us Such a Nice & Helpful Information, please keep writing and publishing these types of helpful articles, I visit your website regularly
    jagranjosh

    ReplyDelete
  44. Thank you for such a nice article keep posting, I am a Regular Visitor of your website.
    ncvt mis home

    ReplyDelete
  45. yojana magazine september 2019 pdf download
    I found your article on Google when I was surfing, it is written very nicely and is optimized .Thank you I visit your website regularly.

    ReplyDelete




  46. Wow What A Nice And Great Article, Thank You So Much for Giving Us Such a Nice & Helpful Information, please keep writing and publishing these types of helpful articles, I visit your website regularly.
    best exercise for burning calories and losing weight

    ReplyDelete
  47. Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me as well as giving.devops Training in Bangalore

    ReplyDelete
  48. Oxigen BC Private Limited Company is India's Largest CSP Provider, which works in all the states of India to open customer service point of all banks. Such as - sbi, boi, bob, pnb etc.

    CSP Apply
    CSP Online Application
    Online CSP Apply
    CSP Registration
    CSP Online Application
    CSP Provider

    ReplyDelete
  49. Very Nice article thank you for share this good article 먹튀검증

    ReplyDelete
  50. The development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. IEEE final year projects on machine learning In case you will succeed, you have to begin building machine learning projects in the near future.

    Projects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.


    Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.

    ReplyDelete
  51. I found your article on Google when I was surfing, it is written very nicely and is optimized .Thank you I visit your website regularly.
    school management software erp

    ReplyDelete
  52. thank you for share this wonderful information 파워볼사이트

    ReplyDelete
  53. This is the exact information I am been searching for, Thanks for sharing the required infos with the clear update and required points. To appreciate this I like to share some useful information.

    mulesoft training in bangalore

    mulesoft courses in bangalore

    mulesoft classes in bangalore

    mulesoft training institute in bangalore

    mulesoft course syllabus

    best mulesoft training

    mulesoft training centers

    ReplyDelete
  54. Really i found this article more informative, thanks for sharing this article! Also Check here

    Download and install Vidmate App which is the best HD video downloader software available for Android. Get free latest HD movies, songs, and your favorite TV shows

    Vidmate App Download

    Vidmate apk for Android devices

    Vidmate App

    download Vidmate for Windows PC

    download Vidmate for Windows PC Free

    Vidmate Download for Windows 10

    Download Vidmate for iOS

    Download Vidmate for Blackberry

    Vidmate For IOS and Blackberry OS

    ReplyDelete
  55. Thank you for the information, I was wondering if you have anther article please notice me on Vegus168 หวยออนไลน์ thank you.

    ReplyDelete
  56. Ombre Powder Glow brows is a technique performed with the assistance of a Master Liner Machine and provides a soft appearance that is both natural looking and long lasting. This brow type typically lasts 1-2 years.

    Microblading
    Permanent Makeup
    Microblading Los Angeles
    Eyebrows
    Eyebrow Tattoo
    Cosmetic Tattoo
    Ombre Eyebrows
    Powder Eyebrows

    ReplyDelete
  57. We have worked with many businesses in New Zealand and abroad and we have found that although there has been massive growth in technology, most small to medium sized business owners have been left behind.

    Website Development Services in New Zealand
    Software Development Services in New Zealand
    E-Commerce Development Services in New Zealand
    Content Writing Services in New Zealand
    Digital Marketing Services in New Zealand

    ReplyDelete
  58. A large number of people, particularly the migrant laborers and factory workers do not have a saving account and even not able to open an account due to lack of valid address and ID proof. As a result they face difficulties to save their earnings in a safe place and look out for solution to send money to their families.

    Apply CSP
    CSP Registration
    CSP Provider
    Bank CSP
    CSP Kisok

    ReplyDelete
  59. Betting games in the Dingdong Gambling Site This organic product is very assorted so betting players simply pick which sort of game is proper with desires and tastes. The principles and arrangements of each game are likewise legitimate and extraordinary so it is sure that the betting game isn't automated in it. For the security strategy, each betting player's record will be appropriately monitored by online openings so there is no should be apprehensive if later there is thievery or infection in the record of every player 우리카지노.

    ReplyDelete
  60. Awesome Writing. Your way of expressing things is very interesting. I have become a fan of your writing. Pls keep on writing.
    CCC Result

    ReplyDelete
  61. thanks for share this amazing article 아바타배팅
    very nice and good article 전화배팅
    thank you for share this amazing article sir 스피드배팅
    great and very good article sir 솔레어아바타배팅
    great and very good article sir 오카다아바타배팅
    great and very good article sir 마이다스아바타배팅

    ReplyDelete
  62. Very nice and great video 아바타배팅
    great and very good article sir 전화배팅
    very nice and amazing article 스피드배팅

    ReplyDelete

  63. thanks for share this amazing article sir 아바타배팅
    very nice and very good article 필리핀아바타배팅

    ReplyDelete
  64. Digital Marketing is a vital Step in establishing a company in Dubai, especially for Startup companies to set up a Strong Business Identity.

    ReplyDelete
  65. amazing content nice thank you Very interesting information blog post, my favorite this post, thanks a lot for these amazing content. https://www.sermseo.com/

    ReplyDelete
  66. Very good post, keep sending us such informative articles I visit your website on a regular basis.
    sr nagar pin code

    ReplyDelete
  67. i like your website your website is very good please keep it up 토토사이트

    ReplyDelete
  68. Really i found this article more informative, thanks for sharing this article! Also Check here 파워볼사이트

    ReplyDelete
  69. Thank you for such a nice article keep posting, I am a RegularVisitor of your website.
    sakshi epaper

    ReplyDelete
  70. I found your article on Google when I was surfing, it is written very nicely and is optimized.Thank you I visit your website regularly.
    wren and martin pdf

    ReplyDelete
  71. I found your article on Google when I was surfing, it is written very nicely and is optimized.Thank you I visit your website regularly.
    temperature in goa

    ReplyDelete
  72. Nice article i was really impressed by seeing this article, it was very interesting and it is very useful for me.I get a lot of great information from this blog. Thank you for your sharing this informative blog..

    Big Data Hadoop Training In Chennai | Big Data Hadoop Training In anna nagar | Big Data Hadoop Training In omr | Big Data Hadoop Training In porur | Big Data Hadoop Training In tambaram | Big Data Hadoop Training In velachery

    ReplyDelete
  73. Finding a dream home is almost every one’s dream and is always an exciting adventure, but at times it
    can get a toll on your mental peace with you becoming stressful and worried all the time.

    commercial property for sale in faridabad
    commercial  property for sale in greater faridabad
    new projects in delhi ncr
    top builders in faridabad
    best investment in Faridabad
    real estate developers  in Faridabad
    construction companies faridabad

    ReplyDelete
  74. thanks for sharing information awesome blog post click here

    ReplyDelete
  75. Really nice and interesting post. I was looking for this kind of information and enjoyed reading this one. Keep posting. Thanks for sharing.
    digital marketing course in guduvanchery

    ReplyDelete
  76. I am so happy to found your blog post because it's really very informative. Please keep writing this kind of blogs and I regularly visit this blog. Have a look at my services.
    Cyber Security Training Course in Chennai | Certification | Cyber Security Online Training Course | Ethical Hacking Training Course in Chennai | Certification | Ethical Hacking Online Training Course | CCNA Training Course in Chennai | Certification | CCNA Online Training Course | RPA Robotic Process Automation Training Course in Chennai | Certification | RPA Training Course Chennai | SEO Training in Chennai | Certification | SEO Online Training Course

    ReplyDelete
  77. Thanks for sharing information awesome blog post Online Education Quiz website For Exam Follow this website Gk in Hindi

    ReplyDelete
  78. This comment has been removed by the author.

    ReplyDelete
  79. Freelance maintenance technicians also benefit from establishing relationships with multiple clients. Working as a freelancer, you’ll gain employment experience with a variety of businesses, boosting your overall skill set and providing you with more work opportunities in the future.

    maintenance technician

    ReplyDelete
  80. Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

    Security Guard License
    Ontario Security License
    Security License Ontario
    Security License

    Thank you..

    ReplyDelete
  81. Generous leads are cardinal for any business, this incorporates little Digital Marketing Company medium and enormous undertakings. The ordinarily realized Digital Marketing Services web-based promoting program that common computerized digital marketing solutions can offer to its clients are Search Engine Optimization, Paid Marketing, online marketing services

    ReplyDelete
  82. I have watched your video and liked it, but it is still have so few likes. I want to help you and share with you site https://soclikes.com/ where I often buy youtube likes

    ReplyDelete
  83. Wonderful blog with great piece of information. Regards to your effort. Keep sharing more such blogs.Looking forward to learn more from you.
    DevOps Training in Chennai

    DevOps Course in Chennai

    ReplyDelete
  84. Excellent blog!!! I got to know more useful information by reading your blog. Thanks for posting this blog.

    Online Training for Big Data
    Big Data Hadoop Online Training

    ReplyDelete
  85. You can easily create custom room in PUBG mobile by just using a single step and it is super easy to create. But however, lots of bugs have been reported by players regarding PUBG mobile mic glitch in the recent time.

    ReplyDelete
  86. Excellent blog thanks for sharing the valuable information..it becomes easy to read and easily understand the information.
    Useful article which was very helpful. also interesting and contains good information.
    to know about python training course , use the below link.

    Python Training in chennai

    Python Course in chennai




    ReplyDelete
  87. Nice information thanks for sharing it’s very useful. This article gives me so much information.

    Java Training in Chennai

    Java Course in Chennai

    ReplyDelete
  88. I was surfing net and fortunately came across this site and found very interesting stuff here. Its really fun to read. I enjoyed a lot. 토토사이트 Thanks for sharing this wonderful information. 먹튀검증

    ReplyDelete
  89. Situs stlouisresumeservice.com merupakan situs agen togel online yang memberikan kalian keuntungan besar dengan semua yang dibutuhkan oleh para pemain. Jadi, bermainlah di situs bandar togel Asia Coloktoto.

    ReplyDelete
  90. Cool stuff you have and you keep overhaul every one of us
    data science course in noida

    ReplyDelete
  91. This is an extremely supportive post, exceptionally educational there is no uncertainty about it. A debt of gratitude is in order for imparting this data to us. I truly value your work.

    ReplyDelete
  92. Thank you for this wonderful post! It has long been extremely helpful. 토토사이트I wish that you will carry on posting your knowledge with us.

    ReplyDelete
  93. Of course, your article is good enough, 먹튀검증 but I thought it would be much better to see professional photos and videos together. There are articles and photos on these topics on my homepage, so please visit and share your opinions.

    ReplyDelete
  94. You are the play of the Mobile Legend Games & you want to HACK MLBB game? So, DOWNLOAD injector tools. EZ Star Injector, EZ Hunter, NC Injector, Marjotech PH , NEW I-MOBA, NEW BoxSkin, YS Patcher

    ReplyDelete