Demonstration: cloud, analytics, mobile and social using IBM Bluemix

This post was originally published on ThoughtsOnCloud on January 3rd, 2015.

In the 2013 IBM Annual Report, IBM Chairman, President and Chief Executive Officer Ginni Rometty talked about the shifts that she sees occurring in industries:

“Competitive advantage will be created through data and analytics, business models will be shaped by cloud, and individual engagement will be powered by mobile and social technologies. Therefore, IBM is making a new future for our clients, our industry and our company…As important as cloud is, its economic significance is often misunderstood. That lies less in the technology, which is relatively straightforward, than in the new business models cloud will enable for enterprises and institutions…The phenomena of data and cloud are changing the arena of global business and society. At the same time, proliferating mobile technology and the spread of social business are empowering people with knowledge, enriching them through networks and changing their expectations.”

Consequently, IBM is focusing on three strategic imperatives: data, cloud and engagement.

Recently I have been demonstrating this in a holistic way by showing the fast deployment of an application running on IBM Bluemix, the company’s platform as a service (PaaS) offering. It’s a social analytics application that provides cross-selling and product placement opportunities enabling systems of engagement with systems of insight. It analyzes social trends like tweets to help sales, marketing and operational staff to target specific customers with more personalized offers using mobile technologies.

Take a look at this video of my demonstration:

How governments can tap into cloud and Internet of Things

This post was originally published on ThoughtsOnCloud on October 2nd, 2014.

In my previous post, I discussed some of the innovations that can be achieved by governments using cloud. I presented on this topic recently at the Westminster eForum Keynote Seminar: Next steps for cloud computing. At the session I went on to explore mobile, the Internet of Things and some changes in the skills needed for cloud.

The session abstract asked the following question about mobile: As device processing power increases, yet cloud solutions rely less and less on that power, is there a disconnect between hardware manufacturers and app and software developers? I think this misses the point. Cloud isn’t about shifting the processing power from one place to another; it’s about doing the right processing in the right place.

U.S. Marine using GPS capabilities on a handheld device to check his location.

At IBM, we talk about the nexus of forces of cloud, analytics, mobile and social (CAMS) and we split information technology (IT) into Systems of Record and Systems of Engagement.

The Systems of Record are the traditional IT—the databases that we’re talking about moving from the existing data centers to the cloud. And, as I’ve discussed in a previous post, moving into the cloud means that we can perform a lot of new analytics.

With mobile and social we now have Systems of Engagement. We have devices that interact with people and the world. Because of their fantastic processing power, these devices can gather data that we’ve never had access to before. For example, these devices make it really easy to take a photo of graffiti or a hole in the road and send it to the local council through FixMyStreet in order to have it fixed. It’s not just the additional processing power; it’s the new instrumentation that this brings. We now have a GPS location so the council knows exactly where the hole is.

In a case I discussed in my previous post, this would have made it a lot easier to send photos and even videos of Madeleine McCann to a photo analytics site, to assist in the investigation of her disappearance in 2007.

We’re also working with Westminster council to optimize their parking. The instrumentation and communication from phones helps us do things we’ve never done before, moving us onto the Internet of Things and making it possible to put connected sensors in parking spaces.

With connected cars, we have even more instrumentation and possibilities. We have millions of cars with thermometers, rain detection, GPS and connectivity that can tell the National Weather Service exactly what the weather is with incredible granularity, as well as the more obvious solutions like traffic optimization.

Let’s move on to talk about skills. IBM has an Academic Initiative in which we supply software to universities at no cost, and IBMers work with university administrators and professors on the curriculum and even act as guest lecturers. For Imperial College, we’re providing cloud based marketing analytics software as well as data sets and skills, so that they can focus on teaching the subject rather than worrying about the IT. Since computer science curriculum in schools is changing to focus more on programming skills, we can offer cloud based development environments like IBM Bluemix. We’re working with the Oxford and Cambridge examination board on their modules for cloud, big data and security.

Students at the lecture 21440848

To be honest, it’s still hard. Universities are a competitive environment and they have to offer courses that students are interested in rather than ones that industry and the country need. IT is changing so fast that we can’t keep up.

Lecturers will teach subjects that they’re comfortable with and students will apply for courses that they understand or that their parents are familiar with. A university recently offered a course on social media analytics, which you’d think would be quite trendy and attractive, but they only had two attendees. It used to be that universities would teach theory and the ability to learn, and then industry would hire them and give them the skills. Now, things are moving so fast that industry doesn’t have the skills and is looking for the graduates to bring them.

Looking at the strategy of moving to the cloud and the changing role of the IT department, we’re finding that outsourcing the day-to-day running of the technology brings about a change in skills needed. It’s less about hands-on IT and more about architecture, governance, and managing relationships with third party providers. A lot of this is typically offered by the business faculty of a university, rather than the computing faculty. We need these groups to work closer together.

To a certain extent, we’re addressing this with apprenticeships. IBM has been running an apprenticeship scheme for the last four years. This on-the-job training means that industry can provide hands-on training with the best blend of up-to-the-minute technical, business and personal skills. This has been very effective, with IBM winning the Best Apprenticeship Scheme from Target National Recruitment Awards and National Apprenticeship Services and Everywoman in technology.

In summary, we need to be looking at the new things that can be achieved by moving to cloud and shared services; exploiting mobile and the internet of things; and training for the most appropriate skills in the most appropriate way.

How do you think governments should utilize cloud and the Internet of Things? And what changes do you think are needed to equip students for a cloud based future? Please leave a comment to join the conversation.

A Guide to IBM Bluemix Resiliency and Security

This post was originally published on ThoughtsOnCloud on February 7th, 2015.

I’m pleased to say that it was also published for the 20,000 attendees at IBM Interconnect on Feb 26th.

B-yCORyUcAA8Gq1IBM Bluemix is suitable for high performance, high input/output (I/O), high availability or latency-sensitive production applications, as well as development and test deployments. This is due to the IBM Bluemix configuration of Cloud Foundry within its data centers and the underlying strength of the IBM SoftLayer cloud infrastructure platform.

All Bluemix applications have their infrastructure automatically deployed as required and in real time. For example, if an application is dynamically scaled because it requires extra capacity, Bluemix handles it automatically. There is a full web-based management console and programmable management interfaces, which enable completely flexible monitoring of users’ applications.

IBM Bluemix configures Cloud Foundry in a highly available topology within the IBM SoftLayer data center. All Cloud Foundry components have been replicated to avoid any single point of failure (SPOF). These components include Droplet Execution Agent (DEA), Cloud Controller, router, Health Manager and login server. If any component fails it will be restarted within the data center while the remaining components provide continued availability. Other deployments can become available for the purposes of disaster recovery for IBM Bluemix applications.

IBM Bluemix exploits the IBM SoftLayer cloud infrastructure platform, hosted in data centers with Tier 3 resiliency. IBM SoftLayer provides a compelling set of service level agreements (SLAs) which in turn provide a strong platform for IBM Bluemix technology.

IBM Bluemix is able to exploit IBM SoftLayer’s triple network, which isolates public Internet, private application traffic and infrastructure management traffic. Together with highly redundant servers, each of which has five network cards, and the ability to seamlessly integrate with secure client private networks, IBM Bluemix applications benefit from a highly available and resilient network.

A large catalog of application services is available, each of which typically provides an appropriate range of priced service levels. The service plan will document a priced service level as well as the free service tier. While the free tier provides the ability for developers to try out the functional behavior, the priced levels provide increasing operational quality of service. This service plan is fully documented with the details of the service performance and capacity, as well as specifying high availability and disaster recovery options. This flexible service approach enables departments to match their development and operations with the appropriate service plan to ensure the most economical mix of service levels.

The IBM approach to information assurance is to provide evidence according to government security principles. IBM Bluemix and its underlying cloud platform infrastructure, IBM SoftLayer, are designed to comply with these 14 principles for all security elements including people, process and technology.

The IBM SoftLayer cloud infrastructure platform has already demonstrated compliance with SOC2 Type II, EU Safe Harbor, and CSA STAR CAIQ and CCM self-assessments, as well as the ISO 9000 quality assurance standard. These standards represent the ongoing commitment to the European Commission data privacy requirements.

From an engineering and support perspective, IBM Bluemix and its underlying cloud infrastructure technologies undergo continuous rigorous security testing in accordance with IBM Secure Engineering development practices. If a security exposure is identified by IBM or a third party, then IBM Support will use the IBM Product Security Incident Response Team (PSIRT) process to apply appropriate and timely updates to ensure the overall system security and integrity is maintained.

As you can see, the security and compliance offered by Bluemix is attractive and comprehensive. Do you think Bluemix is right for you?

Sam’s Views on Cloud for Government Policy Makers

I was honoured to be asked to present yesterday on “Cloud Skills, Flexibility and Strategy” at the Westminster eForum Keynote Seminar: Next steps for cloud computing.

English: The Palace of Westminster from Whitehall.

English: The Palace of Westminster from Whitehall. (Photo credit: Wikipedia)

As explained on its website, Westminster Forum Projects enjoys substantial support and involvement from key policymakers within UK and devolved legislatures, governments and regulatory bodies and from stakeholders in professional bodies, businesses and their advisors, consumer organisations, local government representatives, and other interested groups. The forum is structured to facilitate the formulation of ‘best’ public policy by providing policymakers and implementers with a sense of the way different stakeholder perspectives interrelate with an aim is to provide policymakers with context for arriving at whatever decisions they see fit.

The abstract to the session asked about the extent to which Government departments embracing the cloud, what progress is being made in achieving the UK’s Data Capability Strategy on skills and infrastructure development, whether organisations are doing enough to address the emerging shortfall in skills and also asked about the contradiction between mobile device power and cloud.

I was part of a panel and the following was my five minute introduction.

In my five minutes I’d like to talk about the power of cloud and within that to address three areas raised in the abstract to this session – shared services and shared data; mobile; and skills.

We see cloud as being used in three different ways – optimisation, innovation and disruption. Most of what I’ve seen so far in cloud adoption is about optimisation or cost saving. How to use standardisation, automation, virtualisation and self service to do the same things cheaper and faster.

What’s more interesting is the new things that can be achieved with the innovation and disruption that this can provide.

I’ve been working with various groups – local authorities, police forces, and universities, discussing consolidating their data centres. Instead of each one managing their own IT environment, they can share it in a cloud. They justify this with the cost saving argument but the important thing is, firstly, that they can stop worrying about IT and focus on what their real role is, and secondly that by putting their data together in a shared environment they can achieve things that they’ve never done before.

English: The road to Welton, East Riding of Yo...

English: The road to Welton, East Riding of Yorkshire, just south of Riplingham. Taken on the Riplingham to Welton road at MR: SE96293086 looking due south. This is typical south Yorkshire Wolds country. (Photo credit: Wikipedia)

For example, Ian Huntley would never have been hired as a caretaker and so the Soham murders would have been less likely to happen if the police force had access to the data that he was known by a different force.

And we wouldn’t have issues with burglars crossing the border between West and North Yorkshire to avoid detection if data was shared.

In Sunderland we predict £1.4m per year in cost savings by optimising their IT environment but what’s more important is that this has helped to create a shared environment for start up companies to get up and running quickly so it’s stimulating economic growth in the area.

Another example is Madeleine McCann. After her disappearance it was important to collect holiday photos from members of the public as quickly as possible. Creating a website for this before cloud would have taken far too long. Nowadays it can be spun up very quickly. This isn’t about cost saving and optimisation, it’s about achieving things that could never have been done before.

This brings me to the question in the abstract about mobile: “As device processing power increases, yet cloud solutions rely less and less on that power, is there a disconnect between hardware manufacturers and app and software developers”. I think this is missing the point. Cloud isn’t about shifting the processing power from one place to another, it’s about doing the right processing in the right place.

English: GPS navigation solution running on a ...

English: GPS navigation solution running on a smartphone (iphone) mounted to a road bike. GPS is gaining wide usage with the integration of GPS sensors in many mobile phones. (Photo credit: Wikipedia)

In IBM we talk about CAMS – the nexus of forces of Cloud, Analytics, Mobile and Social, and we split the IT into Systems of Record and Systems of Engagement. The Systems of Record are the traditional IT – the databases that we’re talking about moving from the legacy data centres to the cloud. And, as we’ve discussed, putting it into the cloud means that a lot of new analytics can happen here. With mobile and social we now have Systems of Engagement. The devices that interact with people and the world. The devices that, because of their fantastic processing power, can gather data that we’ve never had access to before. These devices mean that it’s really easy to take a photo of graffiti or a hole in the road and send it to the local council through FixMyStreet and have it fixed. It’s not just the processing power, it’s the instrumentation that this brings. We now have a GPS location so the council know exactly where the hole is. And of course this makes it a lot easier to send photos and even videos of Madeleine McCann to a photo analytics site.

We’re also working with Westminster council to optimise their parking. The instrumentation and communication from phones helps us do things we’ve never done before, but then we move onto the Internet of Things and putting connected sensors in parking spaces.

With connected cars we have even more instrumentation and possibilities. We have millions of cars with thermometers, rain detection, GPS and connectivity that can tell the Met Office exactly what the weather is with incredible granularity, as well as the more obvious solutions like traffic optimisation.

Moving on to talking about skills. IBM has an Academic Initiative where we give free software to universities, and work with them on the curriculum and even act as guest lecturers. With Imperial College we’re proving cloud based marketing analytics software as well as data sets and skills, so that they can focus on teaching the subject rather than worrying about the IT. With computer science in school curriculums changing to be more about programming skills we can offer cloud based development environments like IBM Bluemix. we’re working with the Oxford and Cambridge examination board on their modules for cloud, big data and security.

Classroom 010

Classroom 010 (Photo credit: Wikipedia)

To be honest, it’s still hard. Universities are a competitive environment and they have to offer courses that students are interested in rather than ones that industry and the country need. IT is changing so fast that we can’t keep up. Lecturers will teach subjects that they’re comfortable with and students will apply for courses that they understand or that their parents are familiar with. A university recently offered a course on social media analytics, which you’d think would be quite trendy and attractive but they only had two attendees. It used to be that universities would teach theory and the ability to learn and then industry would hire them and give them the skills, but now things are moving so fast that industry doesn’t have the skills and is looking for the graduates to bring them.

Looking at the strategy of moving to the cloud, and the changing role of the IT department, we’re finding that by outsourcing the day to day running of the technology there is a change in skills needed. It’s less about hands on IT and more about architecture, governance, and managing relationships with third party providers. A lot of this is typically offered by the business faculty of a university, rather than the computing part. We need these groups to work closer together.

To a certain extent we’re addressing this with apprenticeships. IBM’s been running an apprenticeship scheme for the last four years This on the job training means that industry can give hands on training with the best blend of up to the minute technical, business and personal skills and this has been very effective, with IBM winning the Best Apprenticeship Scheme from Target National Recruitment Awards and National Apprenticeship Services and Everywoman in technology.

In summary, we need to be looking at the new things that can be achieved by moving to cloud and shared services; exploiting mobile and the internet of things; and training for the most appropriate skills in the most appropriate way.

Using a Cloudant database with a BlueMix application

I wanted to learn how to use the Cloudant database with a BlueMix application. I found this great blog post Build a simple word game app using Cloudant on Bluemix by Mattias Mohlin. I’ve been working through it.

image001

I’ve learned a lot from it – as the writer says “I’ll cover aspects that are important when developing larger applications, such as setting up a good development environment to enable local debugging. My goal is to walk you through the development of a small Bluemix application using an approach that is also applicable to development of large Bluemix applications.” So it includes developing on a PC and also setting up Cloudant outside of BlueMix.

So here’s my simplified version focusing purely on getting an application up and running using a Cloudant BlueMix service and staying in DevOps Services as much as possible.

The first step is to take a copy of Mattias’s code so go to the GuessTheWord DevOps Services project.

click on “Edit Code” and then “Fork”

image003

I chose to use the same project name GuessTheWord – in DevOps Services it will be unique as it’s in my project space.

image005

This takes me into my own copy of the project so I can start editing it.

I need to update the host in the manifest file otherwise the deployment will conflict with Mattias’s. So in my case I change it to GuessTheWordGarforth but you’ll need to change it to something else otherwise yours will clash with mine. Don’t forget to save the file with Ctrl-S, or File/Save or at least changing file.

image007

Now I need to set up the app and bind the database on BlueMix so I click on “deploy”. I know it won’t run but it will start to set things up.

At this point I logged onto BlueMix itself for the first time and located the new GuessTheWord in the dashboard.

image009

I clicked on it and selected “add a service” and then scrolled down to the Cloudant NoSQL DB

image011image013

and click on it. I clicked on “create” and then allowed it to restart the application. Unsurprisingly it still did not start as there is more coding to do. However the Cloudant service is there so I clicked on “Show Credentials” and saw that the database has  username, password, url etc so the registration etc on the Cloudant site is not necessary as this is all handled by BlueMix.

image015image017Clicking on Runtime on the left and then scrolling down to Environment variables I can see that these Cloudant credentials have been set up as VCAP_SERVICES environment variables for my app. So I just need to change the code to use these.

I switch back to DevOps Services and go to the server.js file to modify the code for accessing this database.

I change line 27 from
Cloudant = env[‘user-provided’][0].credentials;
to
Cloudant = env[‘CloudantNoSQLDB’][0].credentials;

So we’re providing the high level environment variable not the name or the label.

Unfortunately there is also an error in Mattias’s code. I don’t know whether the BlueMix Cloudant service has changed since he wrote it but he builds the url for the database by adding the userid and password to it but actually these are already in my environment variable url

so I change line 30 from

var nano = require(‘nano’)(‘https://’ + Cloudant.username + ‘:’ + Cloudant.password + ‘@’ + Cloudant.url.substring(8));
to simply
var nano = require(‘nano’)(Cloudant.url);

Now save the file and click deploy. When it’s finished a message pops up saying see manual deployment information in the root folder page.

image019

So I click on that and hopefully see a green traffic light in the middle.

image021

Click on the GuessTheWord hyperlink and should take you to the working game which in my case is running at

http://guessthewordgarforth.mybluemix.net/

image023

However there are still no scores displayed as there is no database table or entries.

I spent a long time trying to do this next part in the code but eventually ran out of time and had to go through the Cloudant website. If anyone can show me how to do this part in code I’d really appreciate it.

So for now, go to the GuessTheWord app on BlueMix and click on the running Cloudant service

image025

From here you get to a Launch button

image027

Pressing this logs you on to the Cloudant site using single sign on

image029

Create a new database named guess_the_word_hiscores. Then click the button to create a new secondary index. Store it in a document named top_scores and name the index top_scores_index. As Mattias says, the map function defines which objects in the database are categorised by the index and what information we want to retrieve for those objects. We use the score as the index key (the first argument to emit), then emit an object containing the score, the name of the player, and the date the score was achieved. Following is the JavaScript implementation of the map function, which we need to add before saving and building the index.

function(doc) {
emit(doc.score, {score : doc.score, name : doc.name, date : doc.date});
}

image031

Again, we should really be able to do the following as part of the program startup but anyway, the following should add an entry to the database, replacing guessthewordgarforth in the URL with the host name you chose for your application:

http://guessthewordgarforth.mybluemix.net/save_score?name=Bob&score=4

You should see a success message. Enter the following URL, again replacing guessthewordgarforth with your application host name.

http://guessthewordgarforth.mybluemix.net/hiscores

The entry you just added should appear encoded in JSON e.g.

[{“score”:4,”name”:”Bob”,”date”:”2014-08-07T14:27:34.553Z”}]

So, the code and the database are working correctly. Now it just remains to play the game. Go to

http://guessthewordgarforth.mybluemix.net

(replacing guessthewordgarforth with your hostname)

This time it will include Bob in the high score table

image033

and click on “Play!”

game

Cloud With DevOps Enabling Rapid Business Development

My point of view on accelerating business development with improved time to market by using lean principles enabled by devops and cloud.