Showing posts with label lambda. Show all posts
Showing posts with label lambda. Show all posts

Thursday, February 20, 2020

Coming from pure RDBMS and SQL world, persisting object states and querying was always a challenge


Now, to give you a background, Java object state can best be represented as JSON.

- remember JPA Entity class definitions
- in pure RDBMS each of Java object would get represented as individual tables
- querying via SQL gets complex

Enter Mongo DB

Natural question 
https://softwareengineering.stackexchange.com/questions/54373/when-would-someone-use-mongodb-or-similar-over-a-relational-dbms

The immediate and fundamental difference between MongoDB and an RDBMS is the underlying data model. A relational database structures data into tables and rows, while MongoDB structures data into collections of JSON documents. JSON is a self-describing, human readable data format. Originally designed for lightweight exchanges between browser and server, it has become widely accepted for many types of applications.

Pros:
  • MongoDB has a lower latency per query & spends less CPU time per query because it is doing a lot less work (e.g. no joins, transactions). As a result, it can handle a higher load in terms of queries per second and is thus often used if you have a massive # of users.
  • MongoDB is easier to shard (use in a cluster) because it doesn't have to worry about transactions and consistency.
  • MongoDB has a faster write speed because it does not have to worry about transactions or rollbacks (and thus does not have to worry about locking).
  • MongoDB does not have a schema in case you have a special use case that can take advantage of that.
Cons:
  • MongoDB does not support transactions. This is how it obtains most of its benefits.
  • In general, MongoDB creates more work (e.g. more CPU cost) for the client server. For example, to join data one has to issue multiple queries and do the join on the client.
  • Even here in 2017 there is less tooling support for MongoDB than there is for relational databases simply because it is newer. There are also fewer MongoDB experts than their relational counterparts.
Points Often Misunderstood:
  • Both MongoDB and relational databases support indexing. Their query performance is similar in terms of executing large queries.
  • MongoDB does not remove the need for migrations or more specifically, updating your existing data as your schema evolves. For example: If you have an application that relies on a users table to contain certain data, and you modify that table to contain different data (let's say you add a profile picture field), then you will still need to either:
    • Write you application to handle objects for which this property is undefined OR
    • Write a one-time migration to put in a default value for this property OR
    • Write code to provide a default value at query time if this field is not present OR
    • Handle the missing field in some other way

Future Links


[1]
short for Bin­ary JSON, is a bin­ary-en­coded seri­al­iz­a­tion of JSON-like doc­u­ments.

http://bsonspec.org/ 

[2]
CRUD operations

https://docs.mongodb.com/guides/server/insert/
https://docs.mongodb.com/guides/server/read_operators/
https://docs.mongodb.com/guides/server/update/
https://docs.mongodb.com/guides/server/delete/

[3]
Query operations in Mongo and analogies with good old RDBMS SQL
https://docs.mongodb.com/manual/tutorial/query-documents/ 

Please note : above link has a good Mongo DB Web Shell to quickly try out commands.
So no need to hunt for VM box and install tarball and put env variable sttings
or pull  a docker image and try out.

You just have to give it try right there
What a way to learn.
LaaS
Learning on Cloud
:)

[4]
Now come in Python
Scripting language to do No SQL Mongo DB operations ( analogical to PL/SQL or .sql files in pure RDBMS world that you did using Oracle Sql Worksheet )

https://api.mongodb.com/python/current/tutorial.html

[5]
New generation needs and reporting and Overview

https://info-mongodb-com.s3.us-east-1.amazonaws.com/MongoDB_Architecture_Guide.pdf

Good read here.
Some experts..
is the best way to create visualizations of MongoDB data anywhere. Build visualizations quickly and easily to analyze complex, nested data. Embed individualcharts into any web application or assemble them into livedashboards for sharing.

Kubernetes Integration

Kubernetes is the industry leading container orchestration platform. It provides you with a consistent automation andmanagement experience anywhere from on-premises infrastructure to the public cloud. Kubernetes users can use theMongoDB Enterprise Operator for Kubernetesthatintegrates with MongoDB Ops Manager to automate andmanage MongoDB clusters. You have full control over yourMongoDB deployment from a single Kubernetes controlplane. You can use the operator with upstream Kubernetes,or with any popular distribution such as Red Hat OpenShiftand Pivotal Container Service (PKS).
[6]

Analogical comparsion with SQL
https://docs.mongodb.com/manual/reference/sql-comparison/


Word of Caution : Do not get carried away by technology and charts.

[7]
Now to integrating Mongo DB queries into Java code
https://www.mongodb.com/blog/post/getting-started-with-mongodb-and-java-part-i

http://central.maven.org/maven2/org/mongodb/mongo-java-driver/


    
        org.mongodb
        mongo-java-driver
        2.12.3
    
 
MongoClient mongoClient = new MongoClient(new MongoClientURI("mongodb://localhost:27017")
 
MongoClient mongoClient = new MongoClient();

Where are my tables?

MongoDB doesn’t have tables, rows, columns, joins etc. There are some new concepts to learn when you’re using it, but nothing too challenging.
While you still have the concept of a database, the documents (which we’ll cover in more detail later) are stored in collections, rather than your database being made up of tables of data. But it can be helpful to think of documents like rows and collections like tables in a traditional database. And collections can have indexes like you’d expect.
DB database = mongoClient.getDB("TheDatabaseName");
 
DBCollection collection = database.getCollection("TheCollectionName");
 
person = {
  _id: "jo",
  name: "Jo Bloggs",
  age: 34,
  address: {
    street: "123 Fake St",
    city: "Faketon",
    state: "MA",
    zip: “12345”
  }
  books: [ 27464, 747854, ...]
} 
 
List books = Arrays.asList(27464, 747854);
DBObject person = new BasicDBObject("_id", "jo")
                            .append("name", "Jo Bloggs")
                            .append("address", new BasicDBObject("street", "123 Fake St")
                                                         .append("city", "Faketon")
                                                         .append("state", "MA")
                                                         .append("zip", 12345))
                            .append("books", books);
 
 
MongoClient mongoClient = new MongoClient();
DB database = mongoClient.getDB("Examples");
DBCollection collection = database.getCollection("people");
collection.insert(person);
 
 
> use Examples
switched to db Examples
> show collections
people
system.indexes
> _ 
  
 
> db.people.findOne()
{
    "_id" : "jo",
    "name" : "Jo Bloggs",
        "age": 34,
    "address" : {
        "street" : "123 Fake St",
        "city" : "Faketon",
        "state" : "MA",
        "zip" : "12345"
    },
    "books" : [
        27464,
        747854
    ]
}
> _
     
[8]
  
Alternatively 
http://zetcode.com/java/mongodb/
 
Enjoy :) 
 
  



Wednesday, February 19, 2020

Exposure to functional programming languages like Python / Scala opens more opps

[1]

Python is among the fastest-growing and most popular programming languages out there today. Here are a few ways to use the coding language across industries.

https://www.techrepublic.com/article/python-5-use-cases-for-programmers/

1. Insurance

Top use: Creating business insights with machine learning
Case study: One American multinational finance and insurance corporation faced competition from smaller companies that were introducing services driven by machine learning. To compete, the insurer allowed teams to develop new applications and services using machine learning; however, with too many sets of data science tools involved, a number of different versions of Python and compatibility issues arose. The company settled on one version of Python to deliver all of the machine learning capabilities needed.

2. Retail banking

Top use: Flexible data transformation and manipulation
Case study: A large American department store chain with an in-store banking arm collects data centrally in a warehouse, and then shares it with multiple applications to enable its supply chain, retail banking, and analytics and reporting needs. While the company standardized on Python for data manipulation, each team created its own version, which created problems. The company decided on a single, standard Python build to increase engineering speed and decrease support costs.

3. Aerospace

Top use: Meeting software system deadlines
Case study: An American multinational aerospace, military, and defense corporation was contracted to provide a number of systems for the International Space Station. While aerospace software focused on critical safety systems is typically written in a language like Ada, those older languages do not lend themselves well to scripting tasks, GUI creation, or data science analysis. Selecting a single Python version offered a larger contract value and no exposure.

4. Finance

Top use: Data mining identify cross-sell opportunities
Case study: An American multinational financial services corporation wanted to mine complex customer and prospect behavioral data as part of a digital transformation project. The company used Python to initiate different data science and machine learning initiatives to examine the structured data it had been collecting for years, and correlated it with unstructured data from the web and social media to increase cross-selling and reclaim resources.

5. Business services

Top use: API access to financial information
Case study: A privately-held financial data and media company had previously provided partners with access to financial information through different electronic resources. Partners wanted to build desktop applications in a variety of languages, including Python, to incorporate the customer's API directly into their own, and created a Python Software Development Kit (SDK) for their financial information API, leading to increased revenue and customer satisfaction.


[2]

https://stackabuse.com/functional-programming-in-python/

Functional Programming is a programming paradigm with software primarily composed of functions processing data throughout its execution. Although there's not one singular definition of what is Functional Programming, we were able to examine some prominent features in Functional Languages: Pure Functions, Immutability, and Higher Order Functions.
Python allows us to code in a functional, declarative style. It even has support for many common functional features like Lambda Expressions and the map and filter functions.
However, the Python community does not consider the use of Functional Programming techniques best practice at all times. Even so, we've learned new ways to solve problems and if needed we can solve problems leveraging the expressivity of Functional Programming.

 [3]
Historical evaluation of python evolution


https://python-history.blogspot.com/2009/04/origins-of-pythons-functional-features.html

"..It is also worth nothing that even though I didn't envision Python as a functional language, the introduction of closures has been useful in the development of many other advanced programming features. For example, certain aspects of new-style classes, decorators, and other modern features rely upon this capability.

Lastly, even though a number of functional programming features have been introduced over the years, Python still lacks certain features found in “real” functional programming languages. For instance, Python does not perform certain kinds of optimizations (e.g., tail recursion). In general, because Python's extremely dynamic nature, it is impossible to do the kind of compile-time optimization known from functional languages like Haskell or ML.
.."


[4]

Tutorial: Python Functions and Functional Programming

https://www.dataquest.io/blog/introduction-functional-programming-python/

In this post, we will:
  • Explain the basics of functional programming by comparing it to object-oriented programming.
  • Cover why you might want to incorporate functional programming in your own code.
  • Show you how Python allows you to switch between the two.
  • The Lambda Expression
  • The Map Function
  • The Filter Function
  • The Reduce Function
  • Rewriting with list comprehensions
  • Writing Function Partials
[5]
 Another link to get started with language syntax

 Concluding thoughts

Getting to know your programming language of choice well by exploring its features, libraries and internals will undoubtedly help you debug and read code faster. Knowing about and using ideas from other languages or programing language theory can also be fun, interesting, and make you a stronger and more versatile programmer. However, being a Python power-user ultimately means not just knowing what you *could* do, but understanding when which skills would be more efficient. Functional programming can be incorporated into Python easily. To keep its incorporation elegant, especially in shared code spaces, I find it best to use a purely functional mindset to make code more predictable and easy, all the while maintaining simplicity and idiomaticity.

[6] Another good and concise one to get started

A practical introduction to functional programming

 

 

 





Followers