Topic: Matt's Speech (Read 5094 times)

JimH · « **on:** January 26, 2014, 09:09:15 am »

As you may know, Matt Ashland, our CTO fell on January 2 and has been in an incapacitated state for three weeks. You can read more here.

He is awake for several hours each day and he can communicate a little by blinking his eyes. He will blink twice for "yes", for example. We know that he can recognize his family and that he knows something about what they are asking. Brooke, his wife, can get him to open his mouth so she can brush his teeth.

But this thread is meant to be about something else. I want to see how we can help him communicate. Any ideas are welcome. I've thought we could teach him Morse Code, for example. But the most promising may be eye tracking software and hardware.

Here is the Wikipedia article:
http://en.wikipedia.org/wiki/Eye_tracking

I've seen two solutions that look promising.

iMotions:
http://imotionsglobal.com/Hardware/

NUIA
A Kickstarter campaign funded a $60 machine that uses the Kinnect camera.
http://www.4tiitoo.com/eye-tracking-gets-kickstarted-4tiitoo-launches-the-nuia-eyecharm-for-kinect/

I'm asking for you to help us think this through so we can order some equipment soon. Thanks.

CountryBumkin · « **Reply #1 on:** January 26, 2014, 10:41:07 am »

It's unbelievable how an intelligent, productive person's life can change so quickly. I know this is just a temporary setback for Matt - but still, it's just unbelievable.
I've been following the postings from Matt's wife at Caringbridge http://www.caringbridge.org/visit/mattashland and I can't imagine how hard this must be on her. Especially since everyday brings new hope, then setback. Matt's temperature is changing from normal to elevated almost every day. This must be very frustrating and freighting for Matt's family.

But that is not what this post about ...
Regarding Matt's condition, does he have enough movement to swipe/touch a finger on a screen (like an iPad)? If so, maybe something like this http://verballyapp.com/index.html

AndrewFG · « **Reply #2 on:** January 26, 2014, 11:08:36 am »

^

I appreciate that you want to DO something Jim, but I would question whether throwing tech at it is the right thing at this time. It sounds like Matt has a lot of hard work ahead of him. Including apparently having to re-learn to talk the old fashioned way. So if you throw him a parallel challenge of learning to talk using a new tech solution, it might overload him and slow him down in the long run. I suggest you have a talk to his therapist before rushing into anything...

iamimdoc · « **Reply #3 on:** January 26, 2014, 12:35:40 pm »

Perhaps a speak and spell or the current equivalent would be of benefit. Not sure about his level of function based on your description but sometimes patients who can't speak are able to write, type, point, etc.

Might be worth a try.

JimH · « **Reply #4 on:** January 26, 2014, 12:36:16 pm »

They've worked out a way to communicate with eye movements so far:

double wink is yes
swipe left or right is no

I am pretty sure he wants to communicate. I'm trying to find a tool.

Awesome Donkey · « **Reply #5 on:** January 26, 2014, 12:37:38 pm »

If he could move his finger, he could have some sort of Stephen Hawking-type of setup for communication.

JimH · « **Reply #6 on:** January 26, 2014, 12:38:29 pm »

Quote from: iamimdoc on January 26, 2014, 12:35:40 pm

Perhaps a speak and spell or the current equivalent would be of benefit. Not sure about his level of function based on your description but sometimes patients who can't speak are able to write, type, point, etc.

He can lift his hand a little occasionally and he can give a thumbs up. But I don't think he would be able to use a finger to press a button yet.

jmschnur · « **Reply #7 on:** January 26, 2014, 01:06:51 pm »

This is a field my wife got her Ph.D in 40 years ago and one my daughter works in her research at Rice. I am told it is likely that it is too early to make a prognosis . It is vital that he be continually stimulated. If he is up to it morse code might do some of that.

Do you know what the MRIs show? Where was/ is the lesion?

Also my wife says it is useful to get a speech therapist to evaluate him ASAP if this has not yet been done.

This will take a long time.

JimH · « **Reply #8 on:** January 26, 2014, 01:37:28 pm »

That's useful. Thanks. I'm not certain where the lesion is. He's missing the back right top of his skull right now. They removed the bone where it broke in order to let his brain have room to swell. They've been trying to put it back this week, but he had fevers that suggested he might have an infection, so they waited. Hopefull, he'll get it back tomorrow or Tuesday.

jmschnur · « **Reply #9 on:** January 26, 2014, 03:27:53 pm »

Always wise to wait on the replacement for lots of reasons. Once you have more information on the nature of the injury to the brain, you can start thinking of computerized aids. But I think it is very early yet , since inflammation still has to subside.

KingSparta · « **Reply #10 on:** January 26, 2014, 09:19:07 pm »

I WAS THINKING MORSE CODE WOULD BE TOO COMPLEX, BUT THEN AGAIN MATT HAS DONE SOME AMAZING THINGS.

Translated: .. / .-- .- ... / - .... .. -. -.- .. -. --. / -- --- .-. ... . / -.-. --- -.. . / .-- --- ..- .-.. -.. / -... . / - --- --- / -.-. --- -- .--. .-.. . -..- --..-- / -... ..- - / - .... . -. / .- --. .- .. -. / -- .- - - / .... .- ... / -.. --- -. . / ... --- -- . / .- -- .- --.. .. -. --. / - .... .. -. --. ... .-.-.-

jmone · « **Reply #11 on:** January 27, 2014, 04:39:55 am »

The guy behind this was up for Australian of the Year - http://www.controlbionics.com/

jmone · « **Reply #12 on:** January 27, 2014, 04:55:42 am »

As others have said, stimulation is the key but given his support group, I don't think this is any area the Matt will have an issue with.

jgreen · « **Reply #13 on:** January 27, 2014, 11:12:28 am »

JimH, I think that right now you want to develop a short-term vocabulary so that he can feel "in touch". This may be with pictograms that he can gesture to (eye movement), or with a morse-like set of signals that reference concepts. For example:

/ / / I feel like nuts today.

/ / I feel better today.

//// I'm cold.

(etc etc etc)

Key here, IMO, is to give him ready access to practical concepts that relate to his hospital environment, and also the latest MC build, of course. BTW, now that his speech is impaired I cannot wait to start arguing with him about the future direction of MC feature sets--but more about that at a later date!

jgreen · « **Reply #14 on:** January 27, 2014, 11:15:34 am »

Having seen "nuts" sustitued for what I really said, make sure that Matt doesn't have to endure a 100% polite and sanitized discourse. This can be one of the most aggravating aspects of verbal incapacitation--everyone talks to you like a Cocker Spaniel.

drmimosa · « **Reply #15 on:** January 27, 2014, 11:52:27 am »

Steve Gleason, former kicker for the New Orleans Saints, was diagnosed with ALS in 2011. He can no longer speak in the conventional sense, but I heard him introduce Pearl Jam at an outdoor concert last fall using an eye tracking interface to speak to the crowd. It was a very moving speech and an amazing concert followed from the band.

It's just an idea, but maybe his organization Team Gleason would be a good place to get information on advanced implementation of eye tracking software.

The team has a website and an active Twitter account:

http://www.teamgleason.org/
http://www.teamgleason.org/technology/
https://twitter.com/TeamGleason

Thanks for taking this on JimH. Our best to Matt and his family.

-Phil

drmimosa · « **Reply #16 on:** January 27, 2014, 11:53:41 am »

Also, the Microsoft Kinect and something like this software might be an affordable place to start, esp. if you know some people with programming skills

.

https://kinecteyetracker.codeplex.com/

NickF · « **Reply #17 on:** January 27, 2014, 02:42:21 pm »

Jim,

From what you have said and from Brooke's reports, it certainly seems that Matt's eyes are the most controllable and agile part of his body at present.

Matt certainly seems aware of what is going on around him and of his own situation. If I were in this situation, I would be seriously frustrated about lack of ability to communicate. Whilst I have no experience of this type of situation, I would suggest not being too ambitious. If you set the bar too high and Matt can't reach it, it will just increase the frustration.

Try the eye tracking but don't make the aim to "talk". Start with a simple test of getting him to follow your finger in all directions. Then tell him where to look. This simple test will tell you just how much control he has. You can then progress to electronic eye tracking. Use it to select simple options at first then gradually increase the "vocabulary". If it doesn't work, you have lost nothing but you could gain something substantial.

I think this is a great idea. Good luck.

Nick.

jgreen · « **Reply #18 on:** January 27, 2014, 03:59:57 pm »

Good point.

However, I was referencing the experience of my father, a Doctor, on his way off the planet. It was immensely frustrating for him to be talked to as if he was a charming little pet, when in fact his impairment was in formulating a verbal response in a time frame appropriate to the conversation.

Yes, professional therapists must diagnose and prescribe a course of treatment, and it is very very early in the recovery. I understood JimH's question to be in regard to establishing an ad hoc rapport, not trying assist in treatment. I am confident that Matt will continue to build on his successes, and that I will get some really plush feature requests out of all this!

llafriel · « **Reply #19 on:** January 28, 2014, 09:49:24 am »

My little sister has CP. She has control over her eyes but little else. Her mind is not impaired though, and she has written several novels and books. She started communicating using a language called bliss. This using a 'flipboard' with a hole in center. There were eight groups around the hole with symbols or words. We could then have eye contact through the flipboard and confirm what group she 'selected' with her eyes. The startpage selected different settings/moods or wordgroups, we then flipped to the page she selected and it had the corresponding words or symbols. The flipboard had the same symbols on both sides of course so we could read directly the word she chose. The flipboard started simple but ended up being quite large as she learned more symbols and words. I don't have a picture of this flipboard atm, but can send you one if you like.
She now has a computer with eye-tracking hardware and software which she communicates and controls peripherals with. She's using mobilephone texting and facebook etc, and controls the tv, doors and such in her home. She still uses the flipboard though for backup and whenever it's more convenient to use.

http://www.tobii.com/

JimH · « **Reply #20 on:** January 28, 2014, 10:11:34 am »

Amazing. Thanks for posting that. Say hello and thank you to your sister.

llafriel · « **Reply #21 on:** January 28, 2014, 10:53:39 am »

Happy to be of some assistance. I will mention this to my sister next time I see her.
Steve Gleason was mentioned, I think he uses tobii equipment. This has been a lifechanger for my sister, but Matt will hopefully recover to a point where this is not needed of course.

Best wishes to Matt and all of you.

JimH · « **Reply #22 on:** January 28, 2014, 04:18:24 pm »

This must be the Bliss that you mentioned.

And here is a pdf introduction to Bliss Symbols:
http://www.crockford.com/blissym/lesson1.pdf

InflatableMouse · « **Reply #23 on:** January 29, 2014, 05:56:41 am »

Grab a whiteboard so you can draw him stuff.

His daughter could play a game with him that way. Watch his eye movement and number of blinks for movement on a playboard for instance. 4 in a row can easily be played. Its a simple game but I suppose communication makes it a little more challenging. Eventually you might even be able to play chess if you manage to establish a reliable way for moving pieces around the board (once Matt's up for that of course).

I think maybe he would appreciate the diversion and the interaction this way could add some fun too.

mykillk · « **Reply #24 on:** January 29, 2014, 07:50:19 pm »

I have mixed feelings about the idea. I can understand wanting to give him more capability to communicate, but understand that he could be experiencing decreased mental functioning. Trying to teach him some complicated new form of communication may only serve to frustrate, not empower, him and that can lead to a sense of futility and depression. It definitely depends on the patient. Perhaps a rehabilitation counselor should be consulted on this?

When he has some control of his fingers, a split keyboard like this might be the way to go. Typing is a skill set deeply engrained in the memory of a techie like Matt. You can get this with 20" of separation, which should be enough to have one half on each side of his body and be able to type even when laying down.

http://www.kinesis-ergo.com/shop/freestyle2-for-pc-us/

JimH · « **Reply #25 on:** January 30, 2014, 02:42:15 pm »

Some good news on the speech front.

Testing by a speech specialist and surgery this afternoon

BenKenobi · « **Reply #26 on:** January 31, 2014, 11:37:24 am »

Having lived with and helped rehabilitate a stroke victim the first step isn't to give them new ways to 'speak'.

Motivation to do things the old way needs to be provided first, step 1 is to identify just how much cognitive ability remains - simple things like a picture of an apple with text on it - Is this an Apple? This tests a number of 'skills' that we take for granted but these could be damaged in some way i.e the ability to read, to recognise objects - clearly Matt seems to be able to recognise family so that's good, he can respond to audio stimuli so that too is good.

Only when you know how much of the 'old Matt' remains can you hope to help and it is too early to know that. The white board idea that's not bad, simple games like 'O's and 'X's - build on the no yes responses - keep the stimulation going.

I'm afraid those around Matt will need to have boat loads of patience, partly because they remember him as he was and will have as much trouble adjusting as Matt does in recovering, Matt will probably be more frustrated than they are, be unable to show it and hence will react in unexpected ways - depends how much of the old Matt is trapped inside trying to reach out but unable to do so.

Probably the best thing that forum members could do right now is to get behind his family, 3 weeks in terms of a brain injury is nothing. Sounds to me as though Matt will need a lot of support going forward, his family and their supporters need to be given the support to keep them going so that they can keep Matt going.

I really don't want to come over as a killjoy but I've lived with a close relative that suffered brain damage through a stroke, I lived with the ups / the downs whilst trying to do the normal day to day things and live my life at the same time. What helped us were those that said - hey we will cut your grass, no let me go for the groceries, look after the children and even provided an ear or just the company to permit the venting of some frustration etc etc.

As for people that feel a need to help Matt and are close enough to do so should support his family, let them take care of Matt - and the Doctors of course.

INTERACT FORUM

Author Topic: Matt's Speech (Read 5094 times)

BenKenobi