In a world where data is increasingly numerous and complex, fuzzy logic appears to be a particularly well-suited information processing method.
By combining artificial intelligence and the modeling of complex systems, it offers promising prospects for organizations seeking to improve the quality of their data and decision-making.
Contents :
What isce fuzzy logic? 🤔
Fuzzy logic is an approach based on a mathematical theory developed in the 60s by mathematician Lotfi Zadeh.
Used today in artificial intelligence, it is a method for processing information processing also known as fuzzy logic. "fuzzy logicto process imprecise or uncertain information.
Unlike classical logic, which states that every proposition is either true or false, fuzzy logic allows a proposition to be expressed with a certain degree of certainty or uncertainty.
Fuzzy logic is therefore an information processing method capable of representing nuances and degrees of concordance, using intermediate values between 0 (no similarity) and 1 (when similarity is total).
Why use fuzzy logic to process your data?
The advantage of fuzzy logic is that it's intuitive, involving variables and nuances in a result that's usually expressed as true or false.
It can therefore be used to resolve complex situations involving the reconciliation of data that are not written in exactly the same way.
It creates bridges between data sources that do not have a common common key. A common key is the sine qua non for cross-referencing data. This opens up a whole new, untapped field of possibilities for organizations using this approach.
To illustrate, fuzzy logic brings together the names of people who have been entered differently, and who cannot be matched by a conventional method of comparison.
Emma Dupont and Emma Dupond
Mylaurie or Malorie or even Mallorie
To discover how fuzzy logic is used in the insurance industry, see our article ''How to get rid of input errors in insurance?''
What are the applications of fuzzy logic? 🔍
Fuzzy logic is used, among other things, in driver assistance systems, pattern recognition, robot control, medical diagnosis and decision-making.
Insurance tools for risk analysis and prevention at as do the artificial intelligences that manage road and air traffic. It can also be found in weather and climate prediction models.
It is particularly useful for bridging the gap between several data sourceswhich may contain elements that are not strictly identical.
When fuzzy logic is applied to a text, it proposes segment correspondencess which can be less than 100%. This makes it possible to avoid typing errors, different encodings, non-identical data structures, etc. A threshold, known as the confidence coefficient, makes it possible to approximate the level of similarity between text segments. A threshold, known as the confidence coefficient, is then used to approximate the level of similarity between the associated text segments.
Fuzzy logic is used in all fields where the need is to process uncertain or imprecise information. Applied to approximate text data, it can be used to match and reconcile non-identical words or groups of words.
Some examples of fuzzy logic applications
To illustrate how fuzzy logic - an essential feature of the Tale of Data solution - can be used to reconcile differently spelled data, here are a few concrete examples of how fuzzy logic is used by our customers.
Fuzzy logic data matching
In all organizations that use databases with contacts / customers / suppliers / members, etc., fuzzy logic can be used to match first and last names despite spelling differences.
Here is a table illustrating an audit carried out by one of our customers on the quality of the data in its customer database and the main errors encountered:
Here's another use case for fuzzy logic.
One of our customers needed to reconcile its purchase orders with the information its suppliers were sending back on orders delivered.
Here are the references of the purchase orders sent to suppliers; these references are composed of 2 letters and 8 numbers:
AX01259783
AT01478936
AX56321454
AO58960123
The suppliers' first transcription error was the confusion between letters and numbers. As purchase orders are made up of letters and numbers, the most common input error was the inversion of the zero with the letter o, or the letter i with the number 1.
What's more, when the first 3 characters included the digit zero or the letter o, there were even more errors, with the latter decreasing the further the zero was from the letters.
Here are a few examples of errors encountered when suppliers enter purchase orders:
Matching by fuzzy logic has enabled :
to reconcile references by combining the zero and the letter o, the i and the 1,
integrate these management rules into reference reconciliation,
no more 'orphan' delivery notes that had to be reconciled by hand, imagining all possible combinations of errors.
In addition to these cases, our customer was able to avoid other errors, such as the deletion of special characters, which had previously prevented him from automatically reconciling references. Here are some of the most frequently encountered examples:
excess hyphens at the beginning or end of a reference,
apostrophes,
extra spaces at the beginning or end of a reference,
upper/lower case on letters,
references entered one after the other, separated only by a comma, with no spaces, ...
Fuzzy logic has enabled our customer to get rid of data entry errors and work with incorrect information, without any impact on their business.
Since errors are inevitable when human beings interact with computer systems, it's reasonable and realistic to consider the battle of input errors lost in advance.
Instead of demanding an improvement in the quality of the data entered by its suppliers, our customer, equipped with Tale of Data, can now concentrate fully on its business; no longer wasting precious time and resources resolving these errors.
Fuzzy logic can therefore be used to reconcile or associate 'almost' identical data, as well as for data enrichment.
Data enrichment using fuzzy logic
What we mean by data enrichmentis the process of supplementing information with other information, by cross-referencing data from different sources.
One of the best-known, simplest and most widely used enrichments is the "V search" function in Excel. In this case, however, it is essential to have a common key between the two sources, so that the data can be reconciled and then cross-referenced.
Fuzzy logic enrichment does away with this common key.
Various fuzzy logic matching strategies are offered as standard in Tale of Data: approximate spelling (one or more differences), phonetics, ignoring case, accents, spaces... Whatever the strategy and function used, Tale of Data detects 'similar' terms and suggests correlations. The user is then free to accept these proposals.
In addition, the weighting of the match by the confidence index measures the reliability of the match. The rate, offered by the Tale of Data solution, can be used as follows:
If index = 1the match is complete and the reconciliation between the sources 100% reliable. In this case, all fields matched are identical.
If the index is between 0.99 and 0.85the proposed combinations are to be studied and the decision taken on a case-by-case basis. For example, there may be only one letter difference (Dupond and Dupont) and yet, despite this difference, they are the same data. So it makes sense to put them together.
Finally, if the if the index is less than 85%the join is unreliable. There are large differences between the reconciled fields, and their study is unlikely to be relevant. Tale of Data allows you not to match these data.
The confidence index facilitates decisions decisions.
To illustrate the fuzzy logic used in data enrichment cases, one of our customers supplements its CRM with public data from the SIRENE database, available in open data.
In his CRM, SIRET or SIREN information is not mandatory, so he can't use this reliable key to bridge the gap between his CRM and the SIRENE database of French companies. He has been forced to use the company name, whose spelling can sometimes be exotic in his CRM.
Fuzzy logic was therefore the essential strategy for associating the company names in the SIRENE database with those in its CRM, whose spellings were not always correct.
For example, here are the different spellings he found in his CRM for the EDF company:
In addition to the enrichment operations performed by fuzzy logic, he was able to identify the duplicates, triplicates and quadruplicates present in his CRM and become aware of the non-quality of his data.
The use of the Tale of Data solution has enabled the company, thanks to its fuzzy logic strategy, not only to enrich its data, but also to improve its data quality (see our article on enriching your data with open data)
Fuzzy logic and Big Data: conclusion
In a nutshell, fuzzy logic is a method for processing information and quality setting to handle approximate or imprecise information.
It is used in many fields to model complex systems and is capable of representing nuances and degrees of correlation between approximate data.
It offers numerous advantages thanks to artificial intelligence, and ensures much broader data combinations than using a common key.
👉 Finally, fuzzy logic guarantees data quality operations with results far superior to reconciliations using a common key. On this subject, see our article on data quality.