Saturday, February 06, 2016

Microsoft’s massive Turing test – are AI teachers on the horizon?

Someone appears on social media and within three days 1.5 million people start chatting to that person, fooling many into thinking they’re human. If you’ve seen the movie HER – this is eerily close to that plot but it actually happened.
The plot thickens, as it appears that Microsoft, in China, has been running a huge Turing experiment. Microsoft’s Chinese, Bing researchers launched Xiaoice (Little Ice) in 2014 on WeChat and Weibo. She can draw upon a deep knowledge (or access to facts) about celebrities, sports, finance, movies… whatever. More than this she can recite poetry, song lyrics and stories, is open, friendly, a good listener, even a little mischievous, funny and chatty. Sentiment analysis allows her to gauge the emotion and mood of the conversation and adapt accordingly.
Conversations
The results were creepy. Within a few days 1.5 million people had conversations with Xiaoice, many went for up to 10 minutes before realizing she was not human. As the software improved, AI techniques, NLP, fed by Bing’s billions of data points and posts, so did the level of conversational engagement. The conversations started to get longer, with an average of 23 exchanges after tens of millions of chats, some go on for hundreds of exchanges. At 0.5 billion conversations and 850,000 followers, who talk to her on average, 60 times a month, Xiaoice has proved to be a very popular companion.
Attribution
Nass & reeves in The Media Equation, a brilliant set of 35 studies, showed that we are gullible, in the sense that we easily attribute human qualities to technology. We easily attribute human intention to tech, so expect, politeness, no awkward pauses and other human qualities in our interface with tech and that’s what tech is only now starting to deliver. Heider’s Attribution Theory also suggests that, in terms of motivation, we attribute external and internal causes to behavior. This we do, not only with humans but increasingly with machines.
Turing test
Xiaoice differs from Watson and other forms of AI, in that she (see how easy it is to slip into gender attribution to a bot) is not trying to solve a problem, like win Jeopardy, or beat the World Champion at chess or GO. Her aim is authentic conversation, or at least conversation that seems authentic to humans. That, in a nutshell, is the Turing test. It may already have been passed, on a massive scale.
AI
Even more astonishing is that as she converses, and the data set grows, she gets better and better. This internal learning feature, typical of such AI techniques, means that she learns, not like a human but to behave like a human. Obviously there is no consciousness here, that is not to say there is no intelligence. That is a philosophical question where the question of consciousness and intelligence may well lead to the idea that all such networks have some form of intelligence, just not that which we know of as human.
Bot Brother
Potentially, there’s a sinister side to this piece of AI driven tech. Big Brother really can be a bot, but a bot designed by governments to do specific jobs, like keep the population under control. Is it any accident that this experiment was run in China, the master of population control? I suspect that they would never have got away with the experiment in any liberal democracy.
Teaching technology

Will it be possible to emulate what teachers do with technology? In many ways it already can and does. Technology will find things out faster and with more accuracy than a human (search). It can hold much more in memory than any human. But this is not about simply emulating teachers subject knowledge, it is also about the wider skills of teaching. Remember also, that teachers are humans, with brains, and brains not only get lots of things wrong, they are full of cognitive biases, often display racial and sexual bias, get tired, need to switch off for around eight hours a day, start to forget and lose their powers. AI does noneof this. Where am I going with this? I’ve been arguing for the last few years that AI is the most important underlying trend in learning technology, as it offers the greatest possibilities for solving the deeper problems in education and learning, such as replication of good teaching, effective feedback, automated assessment, motivation, access and scale. 
We know from recent work, such as Todai and at Stanford, that AI is starting to get very good at educational tasks such as passing exams, essay marking and predicting learner attainment. It is also delivering more effective learning experiences. This is why every major tech company on the planet is pouring money into AI. We did not go from running speed to 100 miles per hour by copying the leg of a cheetah – we invented the wheel. So it is with AI. We are not copying teachers’ brains, we are building things that may turn out to be better. Note that part of this process, with current systems, such as essay marking or beating champions at GO, involves training the system using real experts, the system then starts to teach itself and get netter and better. It’s like CDP on steroids. That’s frightening. This form of AI introduces the possibility the the ‘teacher’ component, a teacher that not only has an enormous knowledge base, but also the human-like skills of being a motivator and respected tutor, may be on the horizon. It’s a distant horizon, nevertheless, it has appeared.

 Subscribe to RSS

Learning Technologies 16; the flat, the sharp & 7 things I would like to have seen

Keynotes – one flat, one sharp
Marshall Goldsmith’s been around for a very long time. I’ve seen him speak before but I’m a little weary of coachy, motivational platitudes. Don’t want to be too grouchy and I know I have to be more positive, be happy at all times, find meaning, be engaged blah blah blah. I found his SIX active questions just plain old :  
Did I do my best to:
1. Be happy?
Nope – happiness is not the state I always aspire to. I know lots of relentlessly happy people. I don’t want to be a clown.
2. Find meaning?
You’re a business coach not a philosopher – your meaning is not my meaning. There may even be no meaning in the universe.
3. Be fully engaged?
No. I often want to disengage. It’s a European thing Marshall – we don’t all want to be earnestly positive and engaged. Often we want to chill.
4. Build positive relationships?
Some of the most interesting people I have known, and know, have been misanthropic, sceptical, grumpy, anti-social and even unpleasant. I gravitate towards people who don’t always pitch their tent in the happy campers area.
5. Set clear goals?
Goals are OK but they can box you in. Many things I’ve achieved in life have been through letting go, taking risks and not sticking to goals. In fact I find people obsessed with ‘goals’ a bit blinkered; education goals, career goals, business goals, social climbing goals etc.
6. Make progress towards goal achievement?
I’d agree with this, if I knew what achievement meant. For lots of people I’ve met it means getting a gong, like an MBE or OBE. That, for me, is playing some sort of establishment game – it diminishes people in my eyes. For others it means being a doctor, lawyer or accountant – but I would have hated being locked into career progression within a ‘profession’.
Marshall was witty but far too prescriptive for me. I’ve never had a coach and quite glad really, as I’ve made my way in the world by largely ignoring the advice of professional coaches and advisors. Get a life, not a coach – that’s my motto.
Ben Hammersly was different and most delegates thought he was sharp and excellent. I’m not a fan of ‘futurists’ but his observations on AI and the rise of powerful consumer tech was compelling. This also happened to be the theme of my own talk on AI in learning, so I’m duty bound to sing his praises. He also has a nice line in direct opposition to Goldsmith, "It's impossible to be productive without being lazy at the same time," and a fine moustache –I like that.
In conclusion on keynotes, one seemed like someone from bygone days, the other seemed more focused on the future.
Conference
I’m not going to comment on the sessions, as I gave one, enjoyed the experience and everyone I saw present tried hard. Many were good, some OK.  I couldn’t attend many but the twitter feed seemed to reflect things well. One observation I would make is the format is too samey. There no real ‘debate’ where issues are thrashed out, I’d have a full on head to head debate at the end of each of the two days.
Exhibition
Having established itself as the most important L&D technology exhibition in the UK (WOLCE, I think, is a dead duck), what were my impressions this year? When I asked people what they thought of the exhibition, I got shrugs of shoulders and ‘nothing much that’s new’ responses. I felt the same. Before the exhibition, the twitter feed was riddled with vendor Tweets, who clearly have numpties doing their marketing. ‘Come visit Stand number XY’ (with big pic of stand), but absolutely no compelling reason in the Tweet to make the trip. That’s the problem right there. Why are you telling me to come see you? What do you have that’s compelling? Where’s the evidence that it works? Why?
LMSs
There’s dozens of LMSs. I’m not in the camp that says the LMS is dead but I am in the camp that says the LMS is not new and more than a little dull. For example, I didn’t see much evidence of xAPI being implemented (although lots of talk). Although there was an excellent session in the conference in xAPI but it remains a dark art and that’s a problem. Let’s not make the same mistakes we did with SCORM. Where was the debate about safe harbour on data that took place on the very days the conference was being held? Julian Stodd had an interesting perspective on this in his blog on LT, when he focused on the concept of big company ‘inertia’.
Content
Lots of people who can make e-learning but simply decorating your cake with the hundreds and thousands that is ‘gamification’ is not enough. ‘Gamification’ in technology based training has been around for 30 years. Few have really grasped the importance of deep game design, many plump for patronising Pavlovian techniques. What I wanted to hear were alternative models around curation and automatically, AI-generated content. What I heard was ‘it works on every device’. Oh yeah – try it on my Apple watch then. The best piece of e-learning I’ve seen this year was by a company that wasn’t even there. The innovations around AI and adaptive learning were noticeably absent (apart from Filtered). A few had VR on the stand but it tended to be stand eye candy. Don’t flash the tech if you don’t know what you want to do with it. I want to see ideas and applications. So, I hear you say, it’s easy to be critical, how about some suggestions, so here goes.
7 things I would liked to have seen/heard about?
1. Training levy
This will hit all of the major employers who attended but nothing mentioned anywhere. People are sleepwalking into the future. This is a serious issue and needs to be unpacked and discussed.
2. Workplace MOOCs
4000 MOOCs and 45 million enrolments, with a massive move by the major vendors towards vocational and workplace learners, yet barely a peep in the conference. That was very odd.
3. Open source
We need a clearer presentation of open source LMS, authoring tools and content. They exists but are often drowned out by the big vendors.
4. Microlearning
It’s not new but there’s a lively debate to be had around chunking and this form of learning. What is it really? How does it relate to the delivery of deep content etc.
5. EdTech
Maybe something on small companies where lots of 5 min pitches could be presented to potential vendors or investors. This is where the real action is.
6. Evidence not bunk
A good session on evidence-based psychology of learning dismissing the bunk. This would have chimed in well with Clive Shepherd’sappeal for more integrity within the L&D community.
7. Millennials
I met a girl at the exhibition and she was in the crowd in the pub in the evening. She was always being paraded as a ‘Millennial’ in L&D but she had insights which we oldies have missed. I feel the need for some learner voices around devices, content etc.
Conclusion
My summary has been as a critical friend, as I’m not in the Goldsmith camp but in the ‘seek, to strive and not to yield’ camp - it’s good but let’s make it better. The things I love about Learning Technologies are; meeting stacks of people, ending up in the pub and coming away with a bunch of new friends and a clutch of conversations that spark off new ideas. As always, it’s the people that matter.

 Subscribe to RSS

Tuesday, February 02, 2016

Is this the best piece of online learning ever?

I’ve designed, developed and delivered lots of online learning solutions, for well over 30 years, but this morning I saw one of the best I’ve ever seen. In many ways it runs against the tide of informal, social, collaborative, micro-learning; as it is an AI-driven, scenario and simulation-based, performance tool.
Chris Brannigan is the CEO of Caspian Learning and unusual in that his background is in neuroscientist. This, along with his technical and financial skills, has allowed him and this team to build a platform that brings the power of flight sims to business. What I saw blew my mind.
Human Performance Intelligence
He uses what he calls ‘Human Performance Intelligence’ to investigate, diagnose and treat business problems within organisations. This can be compliance issues, risk or performance of any kind. The aim is to do a complete health check, using AI-driven, 3D simulated scenarios, sophisticated behavioural analysis right through to predictive analysis and recommendations for process, human and other types of change. The ambition is breathtaking.
Risky business
Let’s take a real example, one that Chris has completed. Financial institutions nearly took us all down in 2008. They still haven’t got their act together and are being fined billions of dollars for regular breaches on risk, processes and misspelling. We know that existing compliance training doesn’t work –in fact we know it’s a tick-box joke. So how do you know that your tens or hundreds of thousands of employees perform under high risk? You don’t. The problem is that the risk is asymmetric. A few bad apples can incur the wrath of the regulators, who are getting quite feisty. You really do need to know what they do, why they do it and what you need to do to change things for the better.
Intelligence to action
The system learns from experts, so that there is an ideal model, then employees go through scenarios (distributed practice) which subtly gathers data over 20 or so scenarios, with lot of different flavours. It then diagnoses the problems in terms of decision-making, reasoning and investigation. A diagnosis, along with a financial impact analysis is delivered to senior executives and line managers, with specific actions. All of this is done using AI techniques that include machine learning and other forms of algorithmic and data analysis to improve the business. It is one very smart solution.
Business improvement
Note that the goal is not to improve training but to improve the business. The data, intelligence, predictive analytics, all move towards decisions, actions and change. The diagnosis will identify geographic areas, cultural problems, specific processes, system weaknesses – all moving towards solutions that may be; more training, investment decisions, system changes or personnel changes. All of this is based on modelling business outcomes. This is a complex business, as it is easy to improve processes, become risk averse and damage your business. The point is to identify an optimum way forward that always increases productivity, while solving other problems.
Conclusion

This ticks all the boxes for me:
  • draws on behaviour of real people
  • sim/scenario-based data gathering
  • focus on actual performance
  • captures expert performance
  • AI/machine intelligence
  • concrete recommendations
  • all about decision making
  • not stuck in 'training' rut
  • direct business impact
  • gets better the more it is used
  • ambitious!
L&D talk a lot about business alignment but often don’t get very far down that track. Chris has succeeded because he moves beyond L&D into other business units. What he gathers for an organisation is a unique data set, combined with a unique platform that really does deliver recommendations for change. It’s light years ahead of happy sheets and Kirkpatrick. What’s more interesting is that it is the polar opposite of much of what is being done at present, with low key, non-interventionist training. I’ll be presenting this as one of a number of AI solution on Thursday 4th Feb at Learning Technologies at London Olympia. See their website.

 Subscribe to RSS

Monday, February 01, 2016

Kids Company – perfect storm of British Amateurism

I’ve spent a lot of time on the Boards of large charities – over 30 years in total. Indeed, I’ve spent half of my working time on arts and education charities for the last ten years, for free. So it didn’t surprise me when I read about the rotten fruit that was the Kids Company. Read this report and despair. Don’t think that this is a one-off, or unusual in any way. It was a perfect storm of what I think lies at the heart of most problems in this area – British Amateurism.
1. Leadership
Surely someone must have seen the problem when it was obvious that the organisation as being run by a bowl of fruit? Even now, Batmanghelidjh’s lack of judgement is so deep that the luxury accommodation, swimming pool, chauffeur and ridiculous expenses seem normal to her. This is someone who couldn’t lead a can-can, yet was allowed to chew through tens of millions of our money without any checks on her behaviour. Part of the problem here is that we are in thrall to an imported idea of charismatic leadership, that is tearing organisations apart. This was a surfeit of self-belief (if not narcissism) paired with a complete lack of management skills.
2. Governor recruitment
Most Governors are NOT recruited objectively. Huge numbers are friends of friends, resulting in a homogeneity of background and outlook. Note that this problem is not solved through some 70s notion of a diversity policy and stuffing your Trustee board with token representatives. That, as we saw in the Kids Co, simply deepened the amateurism. It’s good to have a heterogeneous group but they must have ‘skills’ to do the job. Many others are recruited through those hideous, London-based agencies, such as Odgers, which narrows things down to their truly awful filtering process. I’ve been through this process (as an interviewer) several times, and believe me, they do the very opposite of what they promise – finding the same old people from the same old circuit. In the end, many charities are stuffed with amateurs, looking for a gong, with little experience of company accounts, risk registers and organisational management. That was the case at the Kids Co.
3. Chair
The Chair of a charity has to have keen eye for governance. Yentob was the Chair for 12 years. He clearly does not understand conflicts of interest and deliberately tried to intervene at the BBC when the fruit hit the fan. His statement about ‘descent into anarchy’ showed that he had lost the plot, if he had ever had one. Resignation from the BBC was the only option. It’s this sort of establishment amateurism that got the BBC into such trouble over Saville. What was more worrying was his complete lack of financial skills. You cannot have a Chair who doesn;t really understand the concept of 'reserves' and risk. In many ways he was more culpable than the fruitcake CEO. He should have dealt with her lack of skills but he couldn;t becauese he also lacked those skills.
4. Risks
If you don’t run a Risk Register, and deal with risks at every Trustee meeting, you’re not doing your job. I really believe this and have introduced Risk management systems to some charities I’ve worked in. This is a key management tool that forces you to attend to, and manage, the sort of risks that took the Kids Company down. A glance at a risk register would have exposed the most obvious risks around governance, funding, management, expenses and reserves. The board seemed to lack the necessary skills to do this job.
5. Auditing
Charities do not get a great deal from auditors. There’s not enough fees from the big boys, so you get a formulaic service that often lacks detail and doesn’t really challenge enough, especially on reserves. To be fair the Charity Commission is hopeless on this issue, not being nearly clear enough on the purpose and need for reserves. What happened at the Kids Company, where there were no reserves, was appalling and the auditors should be punished, as should the Governors, who let this slide by year after year.
6. Politicians
Cameron loved this mob, as they fitted his, as it turned out, doomed policy on The Big Society. This was all too cosy but what were Letwin and Hancock playing at when they made these latest payments? Again we see another strand of British amateurism – the public schoolboy. All three went to Eton, where their knowledge of poverty, and the real needs of this charity, was hardly enhanced. They have proved to be inept in the sense of being unable to judge impact. This was all about appearances trumping reality. Politicians love a little bit of colourful,PR and there is no bigger cornucopia of colour than Batmanghelidjh.
7. Charity Commission
Run by the hapless William Shawcross (guess what an old-Etonian), the CEO is from KPMG and it has a surfeit of lawyers on its board, who don’t seem to have a grip on operational matters - lawyers rarely do. This is an inept, amateurish, toothless organisation. Charities are running amok, with poor governance, no policing and when found out, the Charity Commission appears has no teeth – it doesn’t even seem capable of gumming them to death.
Conclusion

Every facet of British amateurism was at work here; recruitment, chair, governance, finance, risks, auditors, the Charity Commission and politicians. There are many charities that are well run with good people doing good work, usually for free, but many should not be charities at all and are run as establishment fiefdoms. 
Camila's Kids Company: The Inside Story will be broadcast on BBC One at 21:00 GMT on Wednesday 3 February, and on BBC Two in Wales.

 Subscribe to RSS