Blog by Tim Cliffe P.G. Dip. (SEN)
Knowledge Management - The Risk to Learning Presented by Data Silos
01 Target Audience(1.1) This article is intended for those having an interest in, or a responsibility for, delivering organisational learning, or the management of organisational knowledge.
02 Executive Summary(2.1) The Organisation Structure is the first point of contact to identify personnel, their roles and responsibilities. The Organisation Structure, together with a RACI profile for each person, is also the foundation for the Knowledge Structure, as it will sign-post the Gate-keepers responsible for various data asset domains.
(2.2) An effective knowledge structure allows everyone to understand where any knowledge/data asset is located, and ensures the results of any knowledge/data searches return every existing relevant data element.
(2.3) The existence of Data Silos must be addressed, and be replaced by a single set of data structures able to communicate with each other. The longer such silos are permitted to exist, the greater the potential for loss of data, and reduction in the organisation’s knowledge base.
- 03 Structure of this Article
- 04. Terminology
- 05. Introduction
- 06. Organisation Structure
- 07. Knowledge Ownership, Recipients, and Gate-keepers
- 08. Knowledge Structures
- 09. Data Silos
- 10. Policies and Procedures
4.1 Organisational learning(4.1.1) That activity, within an organisation, that records and/or delivers new, or pre-existing knowledge for the use of organisational members/associates.
4.2 Knowledge Management(4.2.1) The structures, policies, procedures, personnel, and information storage/dissemination mechanisms (e.g. IT).
(4.2.2) Such elements apply structure to information in facilitation of efficient identification and retrieval of information for application to tasks, and/or in support of Organisational Learning.
4.3 Collaboration(4.3.1) In the context of this article, the use of collaborative electronic media in the workplace.
4.4 Personal Silos(4.4.1) Created when personnel hoard digital media, for fear of losing it. This behaviour is encouraged by the lack of effective Knowledge Structures.
4.5 Parent Silo(4.5.1) The IT system of an organisation that has acquired or taken-over another company.
4.6 Inherited Silo(4.6.1) The IT system(s) of a company that has been acquired or taken-over by another (parent) organisation.
05 Introduction(5.1) When discussing the needs of clients, with regard to their learning resource development, the knowledge possessed by the client, to enable the development of resources, is essentially assumed as given. However, as I have discovered, the organisation’s ability to find its own knowledge cannot be assumed as often as one may think.
(5.2) During my work as a learning consultant, I have experienced many working environments, which, to a greater or lesser degree, are challenged by the need to structure ever increasing amounts of information, generated on a daily basis, possibly over many years. This may include knowledge/data stored on inherited collaborative systems resulting from acquisition, or take-over.
(5.3) As time passes, and the amount of information increases, any shortcomings in Knowledge Management conflict with efficient identification and retrieval of organisational knowledge. Further, the resources and time required, in enforcing structure on an increasing diaspora of organisational knowledge, become prohibitive.
(5.4) Sooner or later, the organisation is faced with an, apparently, impossible dilemma.
(5.5) The organisation perceives it lacks the resources to apply structure to its knowledge, but cannot function effectively, on a day-to-day basis, without access to that same organisational knowledge. The challenges presented by such a condition become especially impactful in Research and Development activities, as future developments are founded on past knowledge and experience.
(5.6) Knowledge that cannot be found:
- May as well not exist;
- Requires the re-creation of that knowledge (repeated effort);
- Results in two versions of the knowledge (similar, but not identical) that may, at some future point, result in confusion, and misdirection (loss of data integrity);
- Unnecessarily adds to the size of the knowledge, compounding the diaspora.
(5.8) For those involved in the commissioning and development of learning resources, such as e-learning, blended learning, and the like, ensuring the learning resource captures all the knowledge possessed by the organisation is crucial. However, all the knowledge that exists may not be easily found, and at some point someone will be forced to say, “That will have to do”.
(5.9) This article explores some of the issues contributory to information loss due to failures in Knowledge Management.
06 Organisation Structure(6.1) The published structure of the organisation is a critical resource in ensuring anyone in the organisation can identify relevant personnel, including his/her responsibilities, dependencies, and location.
(6.2) The Organisation Structure must, itself, be intuitively designed to maximize ease and speed of use.
(6.3) The importance of the currency of the Organisation Structure cannot be overly stressed. If personnel learn to distrust the published Organisation Structure, they will stop referring to the document, and this, in turn, will lead to adaptive and unpredictable behaviour - The beginnings of unstructured information.
(6.4) The Organisation Structure must be held on a medium that allows instant up-dating and re-publishing, and must be easily accessible at all times. The Organisation Structure should have the ability to automatically notify personnel, when changes are made.
07 Knowledge Ownership, Recipients, and Gate-Keepers(7.1) Although the title of this section may suggest three distinct aspects, in practice they may be all embodied by one person, across various activities.
(7.2.) The term RACI Matrix may be familiar. This explanation of the RACI Matrix, saves unnecessary duplication here.
(7.3) A RACI Matrix is a structured and effective method for describing an individual’s function in the organisation. A summary of an individual’s RACI Matrix should be included in the Organisation Structure to facilitate the identification of appropriate personnel.
(7.4) The RACI Matrix identifies all knowledge owners, knowledge recipients, and gate-keepers, however, the Gate-keeper, in the context of this article, is a special case.
7.5 Gate-keepers(7.5.1) Gate-keepers are those personnel given the task of ensuring policies and procedures, concerned with the structuring of knowledge (information), are followed.
(7.5.2) The Gate-keeper will review and evaluate the practice of collaborators in the use of data storage structures, such as computer directory trees, file-naming conventions, and meta-data for individual files.
(7.5.3) The Gate-keeper is not necessarily the person responsible for prescribing directory trees, file-naming conventions, or meta-data vocabulary. However, they will be instrumental in formerly identifying areas for improvement.
08 Knowledge Structures
8.1 Introduction(8.1.1) The absence of Knowledge Structures in organisations, large and small, will eventually be disastrous.
(8.1.2) The absence of Knowledge Structures results in:
- Difficulty in identifying relevant personnel;
- Personnel needing to unofficially adopt roles, in the absence of guidance;
- The effective loss of organisational knowledge;
- Personnel needing to behave adaptively, and therefore unpredictably, so as to be able to perform their role;
- The hoarding of knowledge for fear of ‘losing’ it (Personal Silos, a form of data silo-ing);
- Duplication of effort and knowledge/data, which is both wasteful, and contributes to loss of knowledge/data integrity;
- Significant costs to the organisation. Costs which, are often not measured, and often not recognised.
8.2 Costs to the Organisation(8.2.1) Consider a medium sized organisation employing 3000 personnel. Let us assume each member of the organisation spends only 1 hour per week searching for difficult-to-find information, and/or identifying the right person to consult/ inform. Now let us assume every member of the organisation costs £10 per hour to employ, and works 46 weeks per year…
3000 x 1 x 10 x 46 = £1,380,000 per year – wasted!
(8.2.2) Of course, we all know the real cost will be many factors greater. So, who pays for this waste? The organisation pays in staff turnover, staff ill health, reduced productivity, and money. The organisation’s customers pay through higher prices or reduced levels of service.
(8.2.3) You may be shouting at your screen “You did not mention loss of competitiveness!” And you are correct, however, as I have already stated, in my experience, such a circumstance is commonplace. Being commonplace, many organisations are affected by the same problem. The result being, little effect on relative competitiveness.
(8.2.4) So why should we do anything about it, if it does not make any difference? Let’s see:
- No one enjoys paying more money for goods and services than would otherwise be the case;
- We claim to be professionals;
- We claim to care about our environment;
- We claim to be innovative;
- We want to enjoy the work we do, not be frustrated by it five days per week, 46 weeks per year, for 45 years and more;
- We want to be better tomorrow than we are today;
- We need to be more efficient because a recession just destroyed a significant part of our economy;
- We would rather not loose our job because of ‘difficult’ times.
8.3 Designing a Knowledge Structure(8.3.1) You know this is coming, don’t you? This is the tricky bit. There is no such thing as the best Knowledge Structure, but there is an effective structure for your organisation, that will reflect best practice.
(8.3.2) Let us assume you have invested time and resources in producing an Organisation Structure fit for purpose, and everyone knows where it is, and how to find and use it.
(8.3.3) Once you have made the important step of realising you need to design your Knowledge Structure, there are a few things you need to do and know:
- DO get all key senior personnel on-board;
- DO develop a make-a-start strategy (include all levels of personnel);
- DO communicate your intentions and commitment to all personnel;
- DO solicit the tacit engagement of all personnel;
- DO ensure you make something ‘visible’ happen quickly, and regularly communicate progress to all personnel;
- KNOW what the organisation’s leaders think is happening now;
- KNOW what the personnel (the doers) are actually doing (6 and 7 will be different);
- KNOW how the Collaboration systems are being used, and how they communicate with each other (where legacy issues exist, they may not communicate with each other – See 9 Data Silos);
- KNOW (a) the goals of the organisation, and (b) the aspirations of personnel in their role (a and b will not necessarily be the same). BOTH are important;
- KNOW the current and future activities/goals of the organisation.
(8.3.5) In gathering the necessary information, you will conduct face-to-face interviews with key personnel at ALL levels, you may run workshops with local personnel to discuss/clarify significant issues identified, and you may develop telephone interviews and on-line questionnaires to access more remote personnel.
(8.3.6) When planning information-gathering activities, you must ensure, as far as possible, the information given by respondents, and subsequently presented in reports, is made anonymous. Remember, after the process is finished, you can walk away, the people you leave behind still have to work together.
(8.3.7) Having reported your initial findings, those findings must be presented, in a structured manner, to the organisation’s personnel for evaluation. There will be things you have wrong. That’s not a problem, because you are about to be put right. There will be many things that ring true with everyone, warts and all. Honesty and forthrightness are paramount throughout this process, but especially now. If you start using ‘political speak’ you will loose the confidence of the people you need to support the process. If you cannot be honest and forthright, get someone who can.
(8.3.8) You will undoubtedly experience a degree of re-iteration in the processes described at 8.3.3 to 8.3.7 above, until you have as much detail as you can, nailed down.
(8.3.9) There will be things you cannot know, or cannot learn, or that will simply take far to long to fully understand. You must accept these, draw a line, and move-on. Your solution cannot be perfect, but it will be far better than chaos, and you can put in place a system of evaluation and improvement via the policies and procedures you will need to develop.
(8.3.10) Now you have reached a position of understanding, you need to devise a set of vocabularies to describe what the organisation is really doing, in a logical and, hopefully, intuitive manner.
(8.3.11) These vocabularies will define the new Knowledge Structure in your data storage systems, i.e. the directory trees, and how they are named. They will define file-naming conventions, and meta-data to be used to enable the searching of the content of individual documents. They will help you to put in order of priority, when to implement the new structure and on what knowledge/data - this priority should also include schedules or methods for transferring legacy data/knowledge to the new structure.
(8.3.12) You will need someone with appropriate experience to help define your vocabularies. One possible source of support could be someone with, e.g., database design experience, as they will understand the principles of data normalisation, and be able to devise efficient and robust data sets and naming conventions.
(8.3.13) All new knowledge/data created can be made to conform to the new structure, from the outset.
(8.3.14) You have just made a significant improvement. You have stopped the knowledge/data diaspora from getting worse.
(8.3.15) As personnel refer to pre-existing knowledge/data in their day-to-day activities, this information can be made to conform to the new structure on an ad-hoc basis.
(8.3.16) You have just made another significant improvement. You have reduced the size of the knowledge/data diaspora.
(8.3.17) You will continue to reduce the diaspora until the point is reached where all referenced pre-existing knowledge/data has been aligned to the new knowledge structure.
(8.3.18) You are now left with legacy knowledge/data that will be subject to the schedules you have devised for their inclusion in the new structure.
(8.3.19) The process will take time to complete, at least several months, for the regularly used knowledge/data to be made to conform to the new structure. Older knowledge/data may take a year or longer. Remember, the data diaspora you inherited probably took many years to develop.
(8.3.20) It is impossible for this article to be a do-it-yourself a-to-z guide to banishing your organisation’s knowledge/data diaspora. Having taken part in projects aimed at precisely that goal, I appreciate how varied and complex individual organisations can be, and consequently the identified solutions. However, there are useful points of practice here, and with competent support, the solution to your knowledge/data diaspora is not as impossible to find as you may think. Finding the solution is crucial to many aspects of your organisation’s functioning, including the wellbeing and productivity of personnel, and the competitiveness of your company.
09 Data Silos(9.1) I have already made reference to Personal Silos, which, are created when personnel hoard digital media for fear of losing it, probably on their workstation hard-drive, rather than on the network, or worse, keeping a copy on both, and one becomes out-of-date (loss of data integrity).
(9.2) In the context of this section, I specifically refer to the more physical data silos, the individual IT systems that multiply when (typically) medium to large organisations acquire other companies, i.e. the Parent Silo, and the Inherited Silo(s).
(9.3) The systems comprising the Parent Silo are likely to be able to communicate with each other, as they will have evolved over time, under a common controlling authority. Indeed, the Inherited Silo(s) will have evolved in a similar manner, but for a different purpose, and under differing controlling authorities, and therefore it is likely the Parent and Inherited Silos cannot communicate directly with each other. This will be for one, or both of the following reasons:
- There is no common data structure;
- The systems are inherently incompatible.
(9.5) Knowledge/data silos are a serious threat to any organisation’s ability to adapt and re-use learning. Evolution teaches us what happens to anything that fails to adapt in a timely fashion.
(9.6) Whatever the scale of the challenge, action must be taken. Sooner or later, the compounding effect of numerous knowledge/data silos will make it impossible for an organisation to function effectively and efficiently. There are also the implications of owners of Personal Silos leaving the organisation, to be considered.
10 Policies and Procedures(10.1) All policies and procedures should describe a mechanism for the regular review, evaluation, and amendment of the activities they control.
(10.2) Policies and procedures should only exist where absolutely necessary, and only then in a minimal form to achieve the desired end.
10.1 Organisation Structure(10.1.1) There must be clear policies and procedures in place to ensure every member of the organisation is captured by the Organisation Structure, in the required detail, from the day they join the organisation, to the day they leave. This must include a capacity to capture any changes to the responsibilities, dependencies, and location of individual personnel.
10.2 Knowledge Ownership, and Recipients(10.2.1) The RACI Matrix will provide a template to formulate policies and procedures relating to the effective identification of responsibilities, accountabilities, and who needs to be consulted, and kept informed.
(10.2.2) Such policies and procedures should also identify methodologies for capturing changes to the RACI profiles of individuals, and how such information is added to the Organisation Structure.
10.3 Gate-Keepers(10.3.1) As a special case, Gate-keepers should be subject to specific policies and procedures defining their role and responsibilities, and how personnel who submit to a Gate-keeper are to conduct themselves.
10.4 Knowledge Structures(10.4.1) Once vocabularies for the Knowledge Structure have been formulated, policies and procedures should define network functions, naming conventions, meta-data, identify personnel responsible for managing the IT function supporting the Knowledge Structure, and allocate clear ‘rights’ (access, read-write-execute) with regard to digital storage systems.
(10.4.2) It should be noted the meta-data vocabulary should not seek to be exhaustive, as this will result in a vocabulary so large, as to make it practicably impossible to use.
(10.4.3) Meta-data vocabulary should prescribe key descriptors only. Individual personnel should have the freedom and responsibility to include other descriptors as necessary. This may require a training intervention.
10.5 Data silos(10.5.1) Policies should describe how the new Knowledge Structure will be implemented on each existing Data Silo, how data is to be managed during the ‘transfer process’, and any schedules for specific milestones leading to the full integration of the Silo into the ‘new network’.
(10.5.2) It is likely such policies may need to address the means by which current Data Silos will ultimately be de-commissioned.
Use of this Article
Any part, or all, of this article may be copied or 'hyperlinked to' for non-commercial purposes. Any copied content or hyperlink to include the following, please¦
Knowledge Management - The Risk to Learning Presented by Data Silos by Tim Cliffe © 2018
Where use will be for commercial purposes, seek authorisation, including details of proposed use, via the Contact Page.
Return to the Blog Page.
3D Printer Developments
Accessibility on the Web
Anonymous Browsing (computer)
Anonymous Browsing (mobile)
e-Learning Blogs - eLearning Feeds
Global Statistics (on-line)
Multi-media Software (free)
Virtual Private Network (VPN)
Virtual Reality Blog
W3C - Web Standards
(External link, opens in a new tab/window) communications, and mobileVisit The Guardian Project web site. Protect the privacy of your mobile on-line browsing.
(External link, opens in a new tab/window) and staticVisit the Tor Project web site. Protect the privacy of your computer on-line browsing.
(External link, opens in a new tab/window) browsing.
Secure your Internet connection with a Virtual Private NetworkVisit the CyberGhost web site.
(External link, opens in a new tab/window) (VPN).
Victim of UK cyber crime? Report itVisit the ActionFraud web site
(External link, opens in a new tab/window).
A great place to start, when looking for any Open Source software, is SourceForgeVisit the SourceForge web site
Open Source software development and distribution community (External link, opens in a new tab/window).