Showing posts with label PubNub. Show all posts
Showing posts with label PubNub. Show all posts

Saturday, 9 April 2016

Live US Jobs Stream

Live US Jobs Stream

Live Jobs 

Take a look at my web page showing a live stream of jobs in North America.

Jobs Stream and Description
On the right side of the page there is a live stream of jobs found in Twitter messages. The log is refreshing automatically once new jobs are found.

See it all here.

Mechanics



Service is working using PubNub queue as source of messages. Python script subscribes to PubNub queue and listens for twitter messages. Once message comes it asks GEO Clustering service to find which cluster a message belongs to. If message belongs to North America cluster it asks text classification service if it contains a job offer. If there is a job offer in it the message it is published to output queue.

We can now subscribe to output queue consume job offers showing them on the web page. There can be many consumers subscribed to the queue.

Conclusions

  • ML engines talking to PubNub queues allow to build chains of services in very easy way.
  • Any kind of text classification is available at hand. It allows for many use cases related to finding information in streams of data.
  • Any kind of clustering is available at hand. GEO heat maps are very easy to implement.

Next

An engine finding sentiment information could be very useful. Stay tuned. Adding it to the chain will give a lot of flexibility.

Sunday, 14 February 2016

Text Classification as a Service

Text Classification as a Service

Let us imagine we have a service collecting unstructured textual data from our partners. We are collecting that data and building service directories out of it.
How to keep our data clean and tidy without investing lots of money in expensive MDM platforms? We can use text classification services.

Text classification services using machine learning technologies keep track of incoming data and help categorize it in fully automatic way. They are using advances text matching algorithms to correlate and clean data.

Text Classification Engine

TC Services Availability


You can have your our own text classification service on demand. Service will be delivered via PubNub queue. It can be started up in minutes and serve your needs just as long as you wish.

Services are built on top of PredictionIO technology and are using PubNub queues as a transport medium. Core components of the engines in most cases are open source. They are in form of templates developed by growing community of PredictionIO developers.

Text Classification Engines are running as Docker containers. This technology allows to create new instances of engines just in minutes in any environment running Docker service. I means they can run in AWS cloud, locally in your back-end servers or even on you laptop running Linux VM.

What do you need to have your own Text Classification Service?

Conclusions

  • Text classification services are somewhere out there in the cloud. But they can be yours with very little effort.
  • You don't need your hardware to get text classified. You can just order an classification engine for you and use it using PubNub queues.
  • Such distributed services scale together with business growth. Cloud does not have borders and limits and you can have as many engines as you can imagine.

Resources

Tuesday, 2 February 2016

Recommendation Engine in Docker Container!

Recommendation Engine in Docker Container!

Check out my docker container with recommendation engine serving recommendations via PubNub queues!

You can find introduction to the idea of Subscribe-Serve and Subscribe-Get Service in my previous post Recommendation as a Microservice.

Show Time

How to run your own recommendation micro-service? It's very easy. You can do it in just few simple steps.

Steps Summary

1. Pull docker image.
2. Start docker container.
3. Train.
4. Get service.

Detailed Instructions

1. Pull docker image.

docker pull goliasz/raas-micro:1.1

2. Create your account and first queue in PubNub.
  • Go to PubNub home.
  • Register. The simples way is just by using Google account.
  • Create you PubNub App.
PubNub application with publish and subscribe keys assigned
Once you have your PubNub application you have your Publish Key and Subscribe Key assigned. 

  • Start Debug Console

Debug console before adding clients
  • Choose your channel name
  • Add two clients
Two queue clients added. First maximized.
3. Start you docker container using your Subscribe Key, Publish Key and Channel ID.

docker run -dt --hostname reco1 --name reco1 -e "PN_PUBKEY=pub-c-1113-demo-3" -e "PN_SUBKEY=sub-c-1f1a-demo" -e "PN_CHANNEL=Channel-mydemo-154" goliasz/raas-micro:1.1 /MyEngine/autostart.sh

Wait two minutes and you should see in your PubNub queue readiness messages.

Readiness messages
You should see three messages.
{
  "msg": "training ready",
  "rtype": "info"
}
 
{
  "msg": "query ready",
  "rtype": "info"
}
 
{
  "msg": "service ready",
  "rtype": "info"
} 
   
4. Train your recommender engine with some data.

Copy/Paste one by one training messages below to PubBub client window and "Send" after each message.

{
  "event": "purchase",
  "entityType": "user",
  "entityId": "u1",
  "targetEntityType": "item",
  "targetEntityId": "Iphone 6",
  "rtype": "train"
}
Click "Send"
{
  "event": "view",
  "entityType": "user",
  "entityId": "U 2",
  "targetEntityType": "item",
  "targetEntityId": "Phones",
  "rtype": "train"
}
Click "Send"
{
  "event": "$set",
  "entityType": "item",
  "entityId": "Galaxy",
  "properties": {
        "categories": [
          "Phones",
          "Electronics",
          "Samsung"
        ]
  },
  "rtype": "train"
}
Click "Send"
After sending first message you should see the message repeated in second client window.
Training message repeated in second client window.
 
After sending all training messages
Now you have to instruct the engine to train its recommendation model. Send service message below.

{
  "cmd": "retrain",
  "rtype": "service"
}
Service "Retrain" message sent
Wait 3 or 4 minutes and get your recommendation.

5. Ask for recommendations

Just send query message.

{
  "user": "u1",
  "item": "Iphone 5",
  "num": 5,
  "rtype": "query"
}
Query message and response with recommendation
You should receive message with recommendation.

Example:

{
  "itemScores": [
    {
      "item": "Galaxy",
      "score": 1.3233743906021118
    }
  ],
  "rtype": "response"
}
Congratulations! You have your own recommender engine in Subscribe-Server architecture!

Conclusions

  • Nothing stops you to have your own recommender engine.
  • It is easy!
Do you have any problems? Just call me. Contact details here http://kolibero.eu/contact
 
 (c) KOLIBERO, 2016

Saturday, 30 January 2016

Recommendation as a Microservice

Recommendation as a Microservice

Successful e-business is using recommendations and this has become a standard. Good recommendations can increase conversion rate by significant percentage. In such cases recommendation engines are worth money spent. But they don't need to be expensive.

Low cost recommendations for everybody

Development state of machine learning platforms is significant and situation is mature enough to make recommendation just as a simple and cheap service.

How about exposing recommendation engines via secure queues available to every business and organization. Such queues are already available. Great example of such service are PubNub queues. This is great simple and secure technology designed for new Internet of Things era.

Subscribe - Get Service 


Subscribe - Get Service
How about just subscribing to a secure queue and receiving recommendation in 200 ms? Secure PubNub queues are accessible right now. You can have your own queue in just couple of clicks. What if on the other side of the queue is micro-service delivering recommendations on demand. Your personal instance of recommendation engine. Technology is available right now.

See my recommendation engine demo serving recommendations via queue.

Subscribe - Train

To have recommendations you need to train your engine with data related to your business. How to do it? Again secure queue is a solution. Just subscribe to your queue and publish fully anonymous data (just ids) to your personal instance of recommendation engine.

Training of Recommendation Engine.

Subscribe - Serve

Trained machine learning engine is just subscribing back to your queue and serving personalized recommendations tailored only to your needs.

Subscribe - Serve Recommendations

The engine can be deployed anywhere. In-house or in the cloud like Amazon's AWS. It can be just rented for some time or just for a finite number of recommendations. Deployment models are fully flexible. Just imagine your solution!

Endless Possibilities


And now! What if we need another service? If we want to answer another question? Some examples. How long is my customer going to stay with me? When is the customer going to buy the product again? What to do to make him/her stay?

Conclusions 

  • Good recommendation is not a luxury. It is available at hand. Just great service at low cost.
  • You don't need data scientists or big IT departments to have it.
  • You can integrate it with your platforms in super easy and secure way.

Resources