Cloud application has several (can discuss separately) advantages . to use those advantages we are moving towards the CLOUD-NATIVE application. Cloud native applications are composed of smaller, independent, self-contained pieces that are called micro-services.
Opposite of CLOUD-NATIVE application is distributed computing which has some 8 fallacies(0-latency,bandwidth is infinite etc) as mentioned in 1994
Salient features
language agnostic
should have different phases- deploy and run
HTTP/REST communication are synchronous in nature and asynchronisation communication adds to loose coupling between services
Language agnostic(free/doubter) comes to forefront ,with standards like AMQP(Advance Messaging Queue Protocol),Message Queuing Telemetry Transport(MQTT) eding out Language specific mechanism like JMS(Java Messaging Service).
A saying goes: be liberal in what you accept and be conservative in what you send. that is if you are defining 4 enumeration possibly generated in API request then if in case the 5th one comes up it should log the error and must not fail the application.
Unit 2: Creating micro-service in java
DEFINITION:
Each micro-service should handle a single complete business function.
API-DESIGN
Two strategies: Top-Down and Bottom-Up approach Mostly used is Top-Down where we first declare the API and then write the corresponding service-code/business code.
SOME NAMING CONVENTION FOR API -DESIGN
Use noun and plural instead of verb
PRACTICES FOR MAINTAINING OF CHANGE IN API:
maintain different versions OR,
pass new version in header
Unit 3 : Locating services
Micro-services are designed to be easily scaled . the scaling is done horizontally by scaling individual services. With multiple-instances of micro-services you need a method to locate the micro-services and load-balancing over different instances of service you are calling.
Following topics involved:
Service registry – consul, zookeeper ,Eureka etc.
Service Invocation -Client-side (sidecar or client Library) or server-side(server proxy)
API gateway
1.Service registry
Service registry is a persistent store that holds all available instances and route on which they can be reached on.
Four reasons why a MICRO-SERVICE must communicate with service-registry:
.Registration – After service deployed. it MUST register with service-registry.
Heart beat – The micro services must send regular heartbeats to show that it is ready to receive requests.
Service discovery – In-order to communicate with other services, micro-service must call service-registry to get the list of available instances.
De-registration – when service is down it must be removed from service-registry
Example : service-registry: consul, Eureka etc. service-registry are registered by registrar(some service registers have internal to them). Which Service-Registry to choose:- Consistency vs Availability: Most service registries provide partition tolerance and either consistency or availability.A service registry cannot provide all three(tolerance,Consistency, Availability) at same time because of CAP theorem. (Banking/Finance related service) choose Consistency Speed(Media Entertainment) choose Availability For example, Eureka provides availability and both Consul and Apache Zookeeper provide consistency
2.Service Invocation
When a service needs to communicate with another service, it uses the information stored in the service registry. Actual call to micro-service are made either from client-side or from server-side.
Server-Side micro-service calls:
Server-side communication with the micro-service is performed by using a SERVICE PROXY. SERVICE PROXY is either part of service-registry or a separate service. micro-services that wants to communicate make a call to WELL KNOWN End points of service proxy .which is responsible for interacting with service registry and forwarding the request. Load balancing is handled completely on server side.
Disadvantage ofServer-Side micro-service calls : Number of hops is increased. Advantage: Testing is easy and proxy can we mocked and tested.
Client-Side micro-service calls:
There is two parts in this process- getting instance(s) of micro-service from service registry and then it makes request to those services. the micro-service location can be cached on client side to avoid extra path travel in a timeout manner so that if service is down calls don’t fail. the Request is handled in one of following two ways: 1.Client-library 2.Sidecar
Client Library
It is part of micro-service itself what it isolates the logic of remote communication. instead of making new we can use 3rd party provided library(Eureka).Other libraries, such as Netflix Ribbon, provide client-side load balancing.
Sidecar
It is deployed with micro service but runs as a separate process(may be like a config-project corresponding to main project ) A good middle ground between the service proxy and a client library is a sidecar. Sidecars maintain the service registration with the service registry, and usually also perform
client-side load balancing for outbound calls to other micro-services. Because sidecars run in their own process, they are independent of language. Netflix Prana is an open source sidecar that provides the capabilities of the Java based Ribbon and Eureka client libraries to non-Java applications.
3.API gateway
All requests from external client comes to API-gateway endpoints and then the calls redirected to specific micro service.
UNIT 4: Micro-service communication
Following Topics involved :
Synchronous vs Asynchronous communicate
Fault tolerance (resilience)
Synchronous vs Asynchronous communicate
Synchronous communication is a request that requires a response, whether that be immediately or after some amount of time. Asynchronous communication is a message where no response is required.
Previous Understanding: Synchronous communication – one message delivered at a time,and other have to wait until previous once release the resource(thread).
Async support for JAX-RS
Although JAX-RS requests will always be synchronous (you always require a response), you can take advantage of asynchronous programming models. The asynchronous support in JAX-RS 2.0 allows threads to perform other requests while waiting for the HTTP response.
Another option is using reactive libraries like RxJava1. RxJava is a Java implementation of ReactiveX2, a library that is designed to aggregate and filter responses from multiple parallel outbound requests.
Asynchronous messaging (EVENTS)
A request from an external client generally must go through several micro-services before returning a response. If each of these calls is made synchronously, then the total time for the call holds up other requests. The more complex the micro service system and the more interactions between micro-services required per external request, the bigger the resultant latency in your system. If the requests made in the process can be replaced with asynchronous events, then you should implement the event instead.
The microservices pattern requires each microservice to own its own data. This requirement means when a request comes in, there might be several services that need to update their databases. Services should advertise data changes by using EVENTS that other interested parties can subscribe to.
To coordinate the messaging of your application, you can use any one of the available message brokers and matching client libraries. Some examples that integrate with Java are the AMQP broker6 with the RabibitMQ Java client7, Apache Kafka8, and MQTT9, which is aimed at the Internet of Things space.
In brief: for better throughput or better response time for ALL THREADS we go by Async Communication.
Internal messaging in Micro-service component
Events can be used within a single microservice. Using internal events can increase the speed of your response to synchronous requests because you do not have to wait for other synchronous requests to complete before returning a response. TTake advantage of the Java EE specification support for events. Internal asynchronous events can be done by using Contexts and Dependency Injection (CDI)1.
Possible Application/Conclusion : To attain responsiveness from the collective APIs introduce MessageBroker(ACtiveMQ,Kafka) .and increase the no. of threads consumers to more than 1. ,because when all the thread will be added to queue and only 1 thread is picked at a time then rest have to wait. But the incoming Thread don’t have to wait it is put in queue.
It is Kind of partial-utilization of Async feature.(because it says 2 things – the new incoming thread should not have to wait for previous one and already placed should be responding – in this case the new incomming don’t have to wait they are put in queue but collective responsiveness is lacking).
When to use Async and Sync ?
Example : 1.GameOn! Suppose a user is authenticated to play the game.he sends the request to update room(to accommodate new player) to a service say-mapService. the user is expecting a response whether the request is updated or not. In this case the Synchronous communication (REST) should be used. Now the DB has been updated for mapService other micro-services needs to be updated of this change this require no response so it can be done by Event(Asynchronous communication).
In Brief: Async offer fast processing but synchronous is easy and simple to implement.
Example : 2.On-line retail store. suppose a user is authenticated and places the Order . the order request to a service needs to respond whether it is successful or not so it’s is synchronous and then the other services which subscriber to this event (like inventory management or the sipping details)will need to update it’s db they don’t require the responding back so these are should be Asynchronous communication.
Fault tolerance
A strong case of micro-service architecture is fault-tolerance and resilience.
Resilient to change:
Consumer OF API:
As a Consumer of API you should Accept unknown attributes Do not issue an exception if you receive an unexpected variable. If the response contains the information you need, it does not matter what else is provided alongside. But validate the request against the variables or attributes that you need
Do not validate against variables just because they are provided. If you are not using them as part of your request, do not rely on them being there.
Producer of API:
Accept unknown attributes as part of the request
Only return attributes that are relevant for the API being invoked
Timeout:
The request must have a timeout. Because we expect services to come and go, we should not be waiting indefinitely for a response. The asynchronous support in JAX-RS 2.0 has a setTimeout function on the AsynResponse object (see “Synchronous messaging (REST)” on page 34). Tools like Failsafe and Netflix Hystrix have built-in support for timeouts.
CIRCUIT-BREAKERS:
If every-time the timeout issue is occurring then it is not a good way to error out each time, we can give fullback after a limited no. of APIs fail . The circuit breaker makes a note every time a specific request fails or produces a timeout. If this count reaches a certain level, it prevents further calls from occurring, instead of returning an error instantly It also incorporates a mechanism for retrying the call, either after a certain amount of time or in reaction to an event.
BulkHeads:
If a compartment of ship fails.it should not sink the whole ship. At each step you should think of removing the dependency.
Search a sorted array by repeatedly dividing the search interval (Divide and conquer )in half. Begin with an interval covering the whole array. If the value of the search key is less than the item in the middle of the interval, narrow the interval to the lower half. Otherwise, narrow it to the upper half. Repeatedly check until the value is found or the interval is empty.
Algorithm – We basically ignore half of the elements just after one comparison.
Compare x(element to be searched for) with the middle element.
If x matches with the middle element, we return the mid index.
Else If x is greater than the mid element, then x can only lie in the right half subarray after the mid element. So we recur for the right half.
Else (x is smaller) recur for the left half.
Two approaches-
Iterative – tip – for simplicity of understanding go with this
Recursive (ignored as slightly complex to understand and implement )
Iterative approach –
int binarySearch(int arr[], int x)
{
int l = 0, r = arr.length - 1;
while (l <= r) {
int m = l + (r - l) / 2;
// Check if x is present at mid
if (arr[m] == x)
return m;
// If x greater, ignore left half
if (arr[m] < x)
l = m + 1;
// If x is smaller, ignore right half
else
r = m - 1;
}
// if we reach here, then element was
// not present
return -1;
}
Class loader sub system: JVM’s class loader sub system performs 3 tasks a. It loads .class file into memory. b. It verifies byte code instructions. c. It allots memory required for the program.
Run time data area: This is the memory resource used by JVM and it is divided into 5 parts a. Method area: Method area stores class code and method code. (metaspace in Java SE 8) b. Heap: Objects are created on heap. c. Java stacks: Java stacks are the places where the Java methods are executed. A Java stack contains frames. On each frame, a separate method is executed. d. Program counter registers: The program counter registers store memory address of the instruction to be executed by the micro processor. e. Native method stacks: The native method stacks are places where native methods (for example, C language programs) are executed. Native method is a function, which is written in another language other than Java.
Native method interface: Native method interface is a program that connects native methods libraries (C header files) with JVM for executing native methods.
Native method library: holds the native libraries information.
Execution engine: Execution engine contains interpreter and JIT compiler, which converts byte code into machine code. JVM uses optimization technique(Hotspot Profiler) to decide which part to be interpreted and which part to be used with JIT compiler. The HotSpot represent the block of code(frequent) executed by JIT compiler.
There are many test runners available for Python. The three most popular test runners are:
unittest
nose or nose2
pytest
PyUnit/ unittest
The one built into the Python standard library is called unittest. In this tutorial, you will be using unittest test cases and the unittest test runner. The principles of unittest are easily portable to other frameworks.
unit-test requires that:
You put your tests into classes as methods
You use a series of special assertion methods in the unittest.TestCase class instead of the built-in assert statement
To convert the test to a unittest test case, you would have to:
Import unittest from the standard library
Create a class called TestSum that inherits from the TestCase class
Convert the test functions into methods by adding self as the first argument
Change the assertions to use the self.assertEqual() method on the TestCase class
Change the command-line entry point to call unittest.main()
Note: Be careful if you’re writing test cases that need to execute in both Python 2 and 3. In Python 2.7 and below, unittest is called unittest2. If you simply import from unittest, you will get different versions with different features between Python 2 and 3.
The Python unit testing framework, sometimes referred to as “PyUnit,” is a Python language version of Junit
unittest module supports test automation, sharing of setup and shutdown code for tests, aggregation of tests into collections, and independence of the tests from the reporting framework.
To achieve this, unittest supports some important concepts:
test fixture
A test fixture represents the preparation needed to perform one or more tests, and any associate cleanup actions.
test case
A test case checks for a specific response to a particular set of inputs. unittest provides a base class, TestCase, which may be used to create new test cases.
test suite
A test suite is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together.
test runner
A test runner is a component which orchestrates the execution of tests and provides the outcome to the user. The runner may use a graphical interface, a textual interface, or return a special value to indicate the results of executing the tests.
A test runner is an object that provides a single method, run(), which accepts a TestCase or TestSuite object as a parameter, and returns a result object. The class TestResult is provided for use as the result object. unittest provides the TextTestRunner as an example test runner which reports test results on the standard error stream by default. Alternate runners can be implemented for other environments (such as graphical environments) without any need to derive from a specific class.
PEP 8 is the Python code style guide, and it sets out rules for things like line length, indentation, multi-line expressions, and naming conventions
1. Pylint
Pylint is a library that checks for PEP 8 style violations and common errors.
can also be run from the command line.
To install, run pip install pylint.
To use Pylint from the command line, run pylint
path/to/dir or pylint [options] path/to/module.py. Pylint will output warnings about style violations and other errors to the console.
You can customize what errors Pylint checks for with a configuration file called pylintrc.
Outsource your code style
Remembering to run linters manually from the command line for each file you change is a pain,
A great solution is to use a library that automatically reformats your code into something that passes PEP 8 for you.
1.Autopep8
Autopep8 automatically formats the code in the module you specify. It will re-indent lines, fix indentation, remove extraneous whitespace, and refactor common comparison mistakes (like with booleans and None).
To install, run pip install --upgrade autopep8.
To reformat code in place, run autopep8 --in-place --aggressive --aggressive <filename>. The aggressive flags (and the number of them) indicate how much control you want to give autopep8 over your code style
Check your test coverage
You’re writing tests, right? Then you will want to make sure new code committed to your codebase is tested and doesn’t drop your overall amount of test coverage.
For measuring test coverage, we have one recommendation: Coverage.
Coverage has several options for the way it reports your test coverage to you, including outputting results to the console or to an HTML page and indicating which line numbers are missing test coverage
To install, run pip install coverage. To run a program and see its output,
run coverage run [path/to/module.py] [args], and you will see your program’s output.
To see a report of which lines of code are missing coverage, run coverage report -m.
Basics of using the logging module to record the events in a file are very simple. For that, simply import the module from the library.
Create and configure the logger. It can have several parameters. But importantly, pass the name of the file in which you want to record the events.
Here the format of the logger can also be set. By default, the file works in append mode but we can change that to write mode if required.
Also, the level of the logger can be set which acts as the threshold for tracking based on the numeric values assigned to each level. There are several attributes which can be passed as parameters.
The list of all those parameters is given in Python Library. The user can choose the required attribute according to the requirement.
After that, create an object and use the various methods as shown in the example.
#importing module
import logging
#Create and configure logger
FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(levelname)s %(message)s'
logging.basicConfig(filename="newfile.log",format= FORMAT,filemode='w')
#Creating an object
logger=logging.getLogger()
#Setting the threshold of logger to DEBUG
logger.setLevel(logging.DEBUG)
#Test messages
logger.debug("Harmless debug Message")
logger.info("Just an information")
logger.warning("Its a Warning")
logger.error("Did you try to divide by zero")
logger.critical("Internet is down")
The above code will generate a file with the provided name
Configuration:
Standard attributes names:
Attribute name
Format
Description
args
You shouldn’t need to format this yourself.
The tuple of arguments merged into msg to produce message, or a dict whose values are used for the merge (when there is only one argument, and it is a dictionary).
asctime
%(asctime)s
Human-readable time when the LogRecord was created. By default this is of the form ‘2003-07-08 16:49:45,896’ (the numbers after the comma are millisecond portion of the time).
Exception tuple (à la sys.exc_info) or, if no exception has occurred, None.
filename
%(filename)s
Filename portion of pathname.
funcName
%(funcName)s
Name of function containing the logging call.
levelname
%(levelname)s
Text logging level for the message (‘DEBUG’, ‘INFO’, ‘WARNING’, ‘ERROR’, ‘CRITICAL’).
levelno
%(levelno)s
Numeric logging level for the message (DEBUG, INFO, WARNING, ERROR, CRITICAL).
lineno
%(lineno)d
Source line number where the logging call was issued (if available).
message
%(message)s
The logged message, computed as msg % args. This is set when Formatter.format() is invoked.
module
%(module)s
Module (name portion of filename).
msecs
%(msecs)d
Millisecond portion of the time when the LogRecord was created.
msg
You shouldn’t need to format this yourself.
The format string passed in the original logging call. Merged with args to produce message, or an arbitrary object (see Using arbitrary objects as messages).
name
%(name)s
Name of the logger used to log the call.
pathname
%(pathname)s
Full pathname of the source file where the logging call was issued (if available).
process
%(process)d
Process ID (if available).
processName
%(processName)s
Process name (if available).
relativeCreated
%(relativeCreated)d
Time in milliseconds when the LogRecord was created, relative to the time the logging module was loaded.
stack_info
You shouldn’t need to format this yourself.
Stack frame information (where available) from the bottom of the stack in the current thread, up to and including the stack frame of the logging call which resulted in the creation of this record.
First, the try clause (the statement(s) between the try and except keywords) is executed.
If no exception occurs, the except clause is skipped and execution of the try statement is finished.
If an exception occurs during execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try statement.
If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown above.
Syntax-
try:
x = int(input("Please enter a number: "))
break
except ValueError:
print("Oops! That was no valid number. Try again..
A try statement may have more than one except clause, to specify handlers for different exceptions. At most one handler will be executed. Handlers only handle exceptions that occur in the corresponding try clause, not in other handlers of the same try statement. An except clause may name multiple exceptions as a parenthesized tuple, for example
except (RuntimeError, TypeError, NameError):
Order or evoking exceptions:
A class in an except clause is compatible with an exception if it is the same class or a base class thereof (but not the other way around — an except clause listing a derived class is not compatible with a base class). For example, the following code will print B, C, D in that order:
class B(Exception):
pass
class C(B):
pass
class D(C):
pass
for cls in [B, C, D]:
try:
raise cls()
except D:
print("D")
except C:
print("C")
except B:
print("B")
Note that if the except clauses were reversed (with exceptB first), it would have printed B, B, B — the first matching except clause is triggered.
Summary: an except clause can capture only those exception class which are Base(or smaller footprint than that of raise exception class)
Supreme Exception Handler:
The last except clause may omit the exception name(s), to serve as a wildcard. Use this with extreme caution, since it is easy to mask a real programming error in this way! It can also be used to print an error message and then re-raise the exception (allowing a caller to handle the exception as well):
import sys
try:
f = open('myfile.txt')
s = f.readline()
i = int(s.strip())
except OSError as err:
print("OS error: {0}".format(err))
except ValueError:
print("Could not convert data to an integer.")
except:
print("Unexpected error:", sys.exc_info()[0])
raise
The try … except statement has an optional else clause, which, when present, must follow all except clauses. It is useful for code that must be executed if the try clause does not raise an exception. For example:
for arg in sys.argv[1:]:
try:
f = open(arg, 'r')
except OSError:
print('cannot open', arg)
else:
print(arg, 'has', len(f.readlines()), 'lines')
f.close()
The use of the else clause is better than adding additional code to the try clause because it avoids accidentally catching an exception that wasn’t raised by the code being protected by the try … except statement.
Both patterns and strings to be searched can be Unicode strings as well as 8-bit strings. However, Unicode strings and 8-bit strings cannot be mixed to match a literal backslash, one might have to write ‘\\\\’ as the pattern string, because the regular expression must be \\, and each backslash must be expressed as \\ inside a regular Python string literal.
The solution is to use Python’s raw string notation for regular expression patterns; backslashes are not handled in any special way in a string literal prefixed with ‘r’. So r”\n” is a two-character string containing ‘\’ and ‘n’, while “\n” is a one-character string containing a newline
Python offers two different primitive operations based on regular expressions: re.match() checks for a match only at the beginning of the string, while re.search() checks for a match anywhere in the string.
Example:
>>> re.match("c", "abcdef") # No match
>>> re.search("c", "abcdef") # Match
however that in MULTILINE mode match() only matches at the beginning of the string, whereas using search() with a regular expression beginning with ‘^’ will match at the beginning of each line.
>>> re.match('X', 'A\nB\nX', re.MULTILINE) # No match
>>> re.search('^X', 'A\nB\nX', re.MULTILINE) # Match
For example, if one was a writer and wanted to find all of the adverbs in some text, he or she might use findall() in the following manner:
>>> text = "He was carefully disguised but captured quickly by police."
>>> re.findall(r"\w+ly", text)
['carefully', 'quickly']
Use case: parsing log file
import sys
import re
logFileName = open(sys.argv[1], "r")
putputFileName = "parsedLogOutput.txt"
outputfile = open(putputFileName, "w")
lines = logFileName.readlines()
regex = "2019-08-12 03:00:*"
compiledReg = re.compile(regex)
for line in lines:
if re.match(compiledReg, line, flags=0):
outputfile.write(line + "\n")
print(line)
Package and making a module deliverable
Each package in Python is a directory which MUST contain a special file called __init__.py. This file can be empty, and it indicates that the directory it contains is a Python package, so it can be imported the same way a module can be imported.
Example:
we create a directory called foo, which marks the package name, we can then create a module inside that package called bar. We also must not forget to add the __init__.py file inside the foo directory.
To use the module bar, we can import it in two ways:
import foo.bar
Tip:Listing functions of module:
# re is a module and following is script to list all functions:
Import re
find_members = []
for member in dir(re):
if "find" in member:
find_members.append(member)
print(sorted(find_members))
Problem Specific code snippets:
1.Run a linux command
make a command
makecmd = "ls -lrt"
# Execute the command
cwd = subprocess.Popen(makecmd, stdout=subprocess.PIPE, shell=True)
(output, err) = cwd.communicate()
# print the output
print("output: " + str(output))
File Handling
Text File
Excel File
There are different modules for handling excel file like :
XlsxWriter(only for writing),openpyxl module(rw all variation of excels),xlwtmodule, panda module
Working with xlwt- in pyhton
Step 1-Install module:
pip install xlwt
Step 2-code snippet to write to an xml and apply style:
Python Pandas is a data analysis library. It can read, filter and re-arrange small and large datasets and output them in a range of formats including Excel.
Note: If only purpose is writing to excel use native module instead of pandas which internally uses it as it will be over-burden.
Pandas writes Excel files using the XlsxWriter modules
This is section is about Interface vs Class and in-depth content on usage of each.
Following rules are described:
Item 15: Minimize the accessibility of classes and members
Item 16: In public classes, use accessor methods, not public fields
Item 17: Minimize mutability
Item 18: Favor composition over inheritance
Item 19: Design and document for inheritance or else prohibit it
Item 20: Prefer interfaces to abstract classes
Item 21: Design interfaces for posterity
Item 22: Use interfaces only to define types
Item 23: Prefer class hierarchies to tagged classes
Item 24: Flavor static member classes over non-static
Item 25: Limit source files to a single top-level class
Highlights of section –
Terms Implementation inheritance- when a class extends another class and Interface inheritance– when a class implements interface or an interface extends another interface.
Loopholes in using Inheritance:
[WRONG Statement]The problems with overriding methods-
Ambiguity in case of lack of Documentation – The case of counting elements of Hashset<>:
there are 2 mehods adding elments to hashset like: add() and addAll()
the case is- that we have to count number of elements in hash set .
we add 3 elements using addall(),and then when we call to user defined function count it RETURNS 6.
Blunder !!!! so internally addAll calls add() method and so it us inserted twcie . this issue occured beasue[Conclusion] there is no documentation for the fact that addAll() call add internally.
[The case of having predicate before adding in a: ]
the problem in inheritance in which method is added not overridden. Suppose you add a method and then in later releases java library also releases method with same signature in super class then your class will be fragile class.
summary :
Use inheritance after aptly analysing if there exists IS-A relationship then class B can extends class A i.e class-B IS-A class-A. Inherit a class only if it made to be inherited else have proper documentation to avoid blunder as we saw in case 1
memory Section for cleanup strategy – permgen vs metaspace
Section 2 :Object Oriented Programming concepts
OOPs Concepts
P.I.E.A
Abstract class & Interface
Working of Abstraction & Interface : syntax and creation
Where to use which entity?
Difference between Abstraction & Interface
Composition Vs Aggregation Vs Association
Section 3 :Collection Framework
Fail-fast & Fail-safe
ArrayList vs CopyOnWriteArrayList
HashMap vs ConcurrentHashMap
Map
Internal Implementation
ConcurrentHashmap vs Syncronized Hashmap
Why use TreeMap ?
Section 4 : ExceptionHandling
final vs Finally vs finalize
Restriction related to exception while overriding methods
ClassNotFoundException vs NoClassDefinationException
ConcurrentModificationException
what if catch and finally both have return statement?
Types of Exceptions
Checked unchecked exception rules during overriding
Compile-time vs Runtime exception
Section 5 : Threads
Immutable Object
why String is immutable
Immutability of Custom Object(string,List<String>)
Why wait/notify/notifyAll is inObject class not in Thread class
Static Synchronised block vs static synchronised method (both are available at class level)
How threads communicate within each other ?
Diff bw :
Volatie and synchronise -can they be used interchanably -No
Atomic variable vs synchronise – can they be used interchangeably – complementary – efficient
Re-entrant lock
CountDownLatch vs Barrier
Wait, synchronized,
Thread-pool (Executor service)and thread local (data specific to each thread)
TBD: optimise synchronised mehtod – 2 method 1 threds gets acces – hold the object? – other thread waiting how to overcome
Section 6 : Java Special purpose classes
Cloneable : Clone Working
Why clone is not part of Cloneable class but of Object class
Can we use clone method without cloneable interface
deep copy and shallow copy
Externisliable vs serialzable (serailID use)
Reflection
Comparable & Comparator
SingBuffer vs StringBuilder
Class Working
Boxing and auto-boxing
Wrapper class use (similar to Adaptor pattern)
Objects accessing
In super class, 2 methods and in-base class 5 methods, how to access base from super
Equals and Hash-code contract
Section 7 : One liners
Section 1 : Internal Working of java
1.JDK Vs JRE VS JVM
What is JRE?
Together,
theJVM
Java API Classes(like util, math, lang, awt,swing etc)
Runtime libraries
form a “platform”/JRE for which all Java programs are compiled.
If you want to run any java program, you need to have JRE installed in the system
– JDK –
JDK contains tools needed to develop the Java programs.
The tools include :
compiler (javac.exe),
Java application launcher (java.exe),
Appletviewer, etc.
Compiler converts java code into byte code. Java application launcher opens a JRE, loads the class, and invokes its main method.
You need JDK, if at all you want to write your own programs, and to compile them. For running java programs, JRE is sufficient.
If u just want to run applets (ex: Online games or puzzles), JRE needs to be installed on the machine.
– JVM –
When we compile a Java file, generates a ‘.class’ file.
‘.class’ file consists of Java byte codes which are understandable by JVM.
JVM interprets the byte code into the machine code depending upon the underlying operating system and hardware combination.
JVM is responsible for all the things like :
garbage collection,
array bounds checking, etc
JVM is platform dependent.
The JVM is called “virtual” because it provides a machine interface that does not depend on the underlying OS and machine hardware architecture. This independence is a cornerstone of the write-once run-anywhere value of Java programs.
There are different JVM implementations. These may differ in things like performance, reliability, speed, etc. These implementations will differ in those areas where Java specification doesn’t mention how to implement the features, like how the garbage collection process works is JVM dependent, Java spec doesn’t define any specific way to do this.
Encapsulation refers to bundling of data,this bundling is provided by class which encapsulates or bundles the fields and methods that manipulate those fields data.
One Point of bundling data is to hide implementation.
Hiding implementation is done by making fields private and there is one more way to hide implementation that is the use of Interface
And By Using Interface the Operations on object are purely based on interface contract.
And this Interface thing provides encapsulation or data hiding in the sense that there is no particular way tied to implement internally(in server side code) the helper methods .
Abstract & Interface
About Abstract Class:
What is Abstract class?
Definition – A class which contains 0 or more abstract methods is known as abstract class. If it contains at least one abstract method, it must be declared abstract.
Can Abstract class have constructor? If Yes then Why is it so?
Yes, Abstract class have the constructor, Since Abstract class can have non-abstract instances and method to initialise them(using constructor chaining ) we need to have constructor though it directly cannot be instantiated.
Why cannot we create an instance of Abstract class? Once a class is abstract it indicates that it may contain incomplete methods hence you cannot create an object of the abstract class.
public abstract class AbstractClassDemo {
abstract void demo();
private int val;
AbstractClassDemo(int a) {
this.val = a;
}
int demo1() {
return val;
};
}
AbstractClassDemo is an abstract class having a non-abstact instance member val,In order to initialise it we need the constructor.
public class ApplicationDemo extends AbstractClassDemo {
ApplicationDemo(int a) {
super(a);
}
public static void main(String[] args) {
ApplicationDemo a = new ApplicationDemo(99);
System.out.println(a.demo1());
}
@Override
void demo() {
// TODO Auto-generated method stub
}
}
Interface vs Abstract class When to use an abstract class and when to use an interface in Java?
Summary Table:
Parameter
Abstract
Interface
Accessibility control
It can be used, if we want to declare non-public members
all methods must be public.
Modifiable
better choice : If we want to add new methods in the future.
if we add new methods to an interface, thenall of the classes that already implemented that interface will have to be changed to implement the new methods.
Versioning
If we want to create multiple versions of our component- create an abstract class. By updating the base class, all inheriting classes are automatically updated with the change
Cannot be changed once created[owing to fact that interface only provides signature and all implementing class must implement it]. If a new version of an interface is required, we must create a whole new interface.
Common functionality
If we want to provide common-implemented functionality among all implementations of our component, use an abstract class. – [can also be provided by any super class not necessarily abstract]
interfaces contain no implementation for any members. exception in java-8 default for backward comparability. Backward comp-ability – if new method added – already implementing class don’t have to override additional methods.
Use-case
Abstract classes should be used primarily for objects that are closely related. Todo: why?
If the functionality we are creating will be useful across a wide range of disparate objects, use an interface. interfaces are best suited for providing a common functionality to unrelated classes.
Inheritance
Diamond problem so multiple inheritance not allowed.Abstract class can extends one other class and can implement one or more interface.
Interfaces are also good when we want to have something similar to multiple inheritances since we can implement multiple interfaces.Interface can extends to one or more interfaces only
Scale
If we are designing large functional units, use an abstract class. Todo: why?
If we are designing small, concise bits of functionality, use interfaces
Level of Implementation
Abstract class is a class that is only partially implemented by the programmer.
Interface provides pure abstraction no implementation
Instantiation
cannot be instantiated
cannot be instantiated
Constructor
can have constructors
can’t have constructors.
Types of methods
can have abstract and non-abstract methods
can have only abstract methods. Since Java 8, it can have default and static methods also. (why?)
Performance
Abstract class is faster than interface
Interface is somewhat slower as it takes some time to find implemented method in class
Both abstract classes and interfaces are tools in Java for achieving abstraction and defining contracts.
Both allow you to declare methods without providing implementations.
Middle layer(Primary differences)
Access modifiers (from table above)
multiple inheritance
Allowed method – abstract vs concrete methods (mention things before java-8 – on prompt bring java-8 addition and reason for addition)
Inner detailed comparison – detailed discussion
Core comparison
Highlight scenarios where you might prefer one over the other based on specific requirements, such as when designing APIs or creating reusable components.
Composition Vs Aggregation Vs Association
The concepts of Association, Aggregation, and Composition describe different types of relationships between classes. These relationships define how objects are connected and how their lifecycles are managed.
Summary Table :
Relationship
UML Symbol
Lifecycle Dependency
Example
Comment
Association
--
Independent
Customer — Payment
A customer can exist even if no payment is made. A payment is linked to a customer but does not “own” it.
Aggregation
◇--
Part can live longer
PaymentGateway ◇– Merchant
Stripe or Razorpay is not “owned” by a merchant. Merchants just use them.
Composition
◆--
Part dies with whole
Payment ◆– AuditTrail Or Human ◆– Heart
Audit trail has no meaning without actual Payment transaction.
Association
A general relationship between two classes.
Objects can be independent of each other.
Can be one-to-one, one-to-many, etc.
Example :
A Payment is associated with a Customer. They exist independently.
class Customer {
private String customerId;
private String name;
// constructors, getters, setters
}
class Payment {
private String paymentId;
private Customer customer; // Association
public Payment(String paymentId, Customer customer) {
this.paymentId = paymentId;
this.customer = customer;
}
}
Aggregation (HAS-A, weak ownership)
A special type of association.
Represents a whole-part relationship, but the part can exist independently of the whole.
Often called a “HAS-A” relationship.
Example :
A Merchant has multiple PaymentGateways, but the gateways (like Razorpay, Stripe) exist independently
class PaymentGateway {
private String name;
// constructors, getters, setters
}
class Merchant {
private String merchantId;
private List<PaymentGateway> gateways; // Aggregation
public Merchant(String merchantId) {
this.merchantId = merchantId;
this.gateways = new ArrayList<>();
}
public void addGateway(PaymentGateway gateway) {
gateways.add(gateway);
}
}
Composition (HAS-A, strong ownership)
A stronger form of aggregation.
If the whole is destroyed, the parts are also destroyed.
The part cannot exist independently.
Example :
Departments can exist outside of the university. If the university is destroyed, departments may still exist.
class AuditTrail {
}
class Payment {
private final AuditTrail adtTrail = new AuditTrail();
}
Section 3 : Collection Framework
Fail-fast and fail-safe
Fail Fast And Fail Safe Iterators in Java
Iterators in java are used to iterate over the Collection objects.Fail-Fast iterators immediately throw ConcurrentModificationException if there is structural modification of the collection. Structural modification means adding, removing any element from collection while a thread is iterating over that collection. Iterator on ArrayList, HashMap classes are some examples of fail-fast Iterator.
Fail-Safe iterators don’t throw any exceptions if a collection is structurally modified while iterating over it. This is because, they operate on the clone of the collection, not on the original collection and that’s why they are called fail-safe iterators. Iterator on CopyOnWriteArrayList, ConcurrentHashMap classes are examples of fail-safe Iterator.
Note- Normal iteration over Data-structure S like – Hash-map/Array-list are fail-fast. there is nothing new about iterator except it traverse over Data-structure
Concurrent Modification: Concurrent Modification in programming means to modify an object concurrently when another task is already running over it. For example, in Java to modify a collection when another thread is iterating over it.
Map
HashMap is one of the most used collection. It doesn’t extend from Collection i/f. BUT collection view of a map can be obtained using keySet,values or entrySet()
Map.Entry interface – static nested interface : This interface represents a map entry (key-value pair).
HashMap in Java stores both key & value object ref – in bucket, As an object of Entry class which implements the nested interface Map.Entry.
How Hash-map internally works? & Internal Implementation
Note: Bucket = i in arr[] : Bucket term used is actually an index of array, that array is called table in HashMap implementation. Thus table[0] is referred as bucket0, table[1] as bucket1 and so on.
Hash-map works on the principal of hashing.
hashCode() working in case of get(k)& put(k,v) operation:
When put() method is used to store (Key, Value) pair, HashMap implementation calls hashcode() on Key object to calculate a hash that is used to find a bucket where Entry object will be stored.
When get() method is used to retrieve value, again key object is used to calculate a hash which is used then to find a bucket where that particular key is stored.
equals() Working :
equals() method is used to compare objects for equality & see if the key is equal to any of the already inserted keys (Recall that there may be more than one entry in the same bucket).
In case of HashMap, key object is used for comparison, also using equals() method Map knows how to handle hashing collision (hashing collision means more than one key having the same hash value, thus assigned to the same bucket. In that case objects are stored in a linked list (grow-able –singly linked)
HashMap provides put(key, value) for storing and get(key) method for retrieving Values from HashMap.
Hash-Collision Summary Table:
Scenario
Behavior
Explanation
Two different keys, same hash index
LinkedList formed
Keys not equal, both stored in chain
Same key inserted again
Value overridden
equals() returns true — replaces
Same bucket has > 8 entries
Treeification starts (if capacity ≥ 64)
LinkedList → Tree for faster lookup
In a nutshell for hash-collision – there are 2scenarios in case of put():
Using hashCode() method, hash value will be calculated. Using that hash it will be ascertained, in which bucket particular entry will be stored. equals() method is used to find if such a key already exists in that bucket,
if equals() on key returns false, then a new node is created with the map entry and stored within the same bucket. A linked-list is used to store those nodes.
If equals() method on key returns true, which means that the key already exists in the bucket. In that case, the new value will overwrite the old value for the matched key.
How get() methods works internally in case of hash-collsion:
As we already know how Entry objects are stored in a bucket and what happens in the case of Hash Collision it is easy to understand what happens when key object is passed in the get method of the Hash-map to retrieve a value.
Using the key again hash value will be calculated to determine the bucket where that Entry object is stored, in case there are more than one Entry object with in the same bucket stored as a linked-list equals() method will be used to find out the correct key. As soon as the matching key is found get() method will return the value object stored in the Entry object.
In case of null Key
As we know that HashMap also allows null, though there can only be one null key in HashMap. While storing the Entry object HashMap implementation checks if the key is null, in case key is null, it always map to bucket 0 as hash is not calculated for null keys.
Hash Map updates in Java 8 Though HashMap implementation provides constant time performance O(1) for get() and put() method but that is in the ideal case when the Hash function distributes the objects evenly among the buckets. But the performance may worsen in the case hashCode() used is not proper and there are lots of hash collisions. As we know now that in case of hash collision entry objects are stored as a node in a linked-list and equals() method is used to compare keys. That comparison to find the correct key with in a linked-list is a linear operation so in a worst case scenario the complexity becomes O(n). To address this issue in Java 8 hash elements use balanced trees(red-black) instead of linked lists after a certain threshold is reached. Which means HashMap starts with storing Entry objects in linked list but after the number of items in a hash becomes larger than a certain threshold, the hash will change from using a linked list to a balanced tree, this will improve the worst case performance from O(n) to O(log n).
Difference between ConcurrentHashMap and Synchronized HashMap:
ConcurrentHashMap
Synchronized HashMap
It is a class that implements the ConcurrentMap and serializable interface.
We can synchronize the HashMap by using the synchronizedMap() method of java.util.Collections class.
It locks some portion of the map.
It locks the whole map.
ConcurrentHashMap allows performing concurrent read and write operation. Hence, performance is relatively better than the Synchronized Map.
In Synchronized HashMap, multiple threads can not access the map concurrently. Hence, the performance is relatively less than the ConcurrentHashMap.
ConcurrentHashMap doesn’t allow inserting null (internally uses HAshTable) as a key or value.
Synchronized HashMap allows inserting null as a key.
What Is the Difference Between Final, Finally, and Finalize in Java?
1. final: Is used to apply restrictions on the class, method, and variable. The final class can’t be inherited — neither method can it be overridden nor variable be changed.
When to use final keyword?
Any field that you never expect to be changed (be that a primitive value, or a reference to an object, whether or not that particular object is itself immutable or not), should generally be declared final. Another way of looking at things is: If your object is accessed by multiple threads, and you don’t declare its fields final, then you must provide thread-safety by some other means.
volatile
synchronised
explicit lock
atomic behaviour
above means are provided for scenarios in which you expect the value of variable/object to change but you want to provide thread safety.
2. finally: this
keyword is used with the try-catch block to provide statements that will always get executed
even if some exception arises. Usually, finally is used to close
resources.
In the presence or absence of excs , finally block ALWAYS
survives(except System.exit(0))
3. finalize: is
used to perform clean up processing just before the object is garbage
collected.
Can You Throw any Exception Inside a Lambda Expression’s Body?
Can Only throw unchecked exceptions, When using a standard functional interface already provided by Java, standard functional interfaces do not have a “throws” clause in its method signatures:
However, if you are using a custom functional interface, throwing checked exceptions is possible.
//Standard Functional Interface
List<Integer> integers = Arrays.asList(3, 9, 7, 0, 10, 20);
integers.forEach(i -> {
if (i == 0) {
throw new IllegalArgumentException("Zero not allowed");
}
System.out.println(Math.PI / i);
});
//Custom Functional Interface
@FunctionalInterface
public static interface CheckedFunction<T> {
void apply(T t) throws Exception;
}
public void processTasks(
List<Task> taks, CheckedFunction<Task> checkedFunction) {
for (Task task : taks) {
try {
checkedFunction.apply(task);
} catch (Exception e) {
// ...
}
}
}
processTasks(taskList, t -> {
// ...
throw new Exception("Something happened");
});
What Are the Rules We Need to Follow When Overriding a Method That Throws an Exception?
When the parent class method doesn’t throw any
exceptions, the child class
method can’t throw any checked exceptions, but it may throw any unchecked.
Here’s an example code to demonstrate this:
When the parent class method throws one or more checked
exceptions, the child class
method can throw any unchecked exception, including all, none, or a subset of
the declared checked exceptions and an even a greater number of these as long
as they have the same scope or narrower. Here’s an example code that
successfully follows the previous rule
ClassNotFoundexception vs Noclassdefinationfounderror
Both occur at run time but in different scenarios and different triggers:
ClassNotFoundException
NoClassDefFoundError
It is an exception. It is of type java.lang.Exception.
It is an error. It is of type java.lang.Error.
It occurs when an application tries to load a class at runtime which is not updated in the classpath.
It occurs when java runtime system doesn’t find a class definition, which is present at compile time, but missing at run time.
It is thrown by the application itself. It is thrown by the methods like Class.forName(), loadClass() and findSystemClass().
It is thrown by the Java Runtime System.
It occurs when classpath is not updated with required JAR files.
It occurs when required class definition is missing at runtime.
Checked Exception (must be handled)
Error (unchecked, serious problem)
ClassNotFoundException –
For example, you may have come across this exception when you try to connect to MySQL or Oracle databases and you have not updated the classpath with required JAR files. Most of the time, this exception occurs when you try to run an application without updating the classpath with required JAR files
public class MainClass
{
public static void main(String[] args)
{
try
{
//OracleDriver not exists in classpath
Class.forName("oracle.jdbc.driver.OracleDriver");
}catch (ClassNotFoundException e)
{
e.printStackTrace();
}
}
}
NoClassDefinationFoundError –
Code-Example :
When you compile the above program, two .class files will be generated. One is A.class and another one is B.class. If you remove the A.class file and run the B.class file, Java Runtime System will throw NoClassDefFoundError.
Class was present at compile time but after build it got removed from target folder
class A
{
// some code
}
public class B
{
public static void main(String[] args)
{
A a = new A();
}
}
Concurrent ModificationException
The ConcurrentModificationException occurs when an object is tried to be modified concurrently when it is not permissible. This exception usually comes when one is working with Java Collection classes.
For Example – It is not permissible for a thread to modify a Collection when some other thread is iterating over it. This is because the result of the iteration becomes undefined with it. Some implementation of the Iterator class throws this exception, including all those general-purpose implementations of Iterator which are provided by the JRE. Iterators which do this are called fail-fast as they throw the exception quickly as soon as they encounter such situation rather than facing undetermined behavior of the collection any time in the future.
Note: It is not mandatory that this exception will be thrown only when some other thread tries to modify a Collection object. It can also happen if a single thread has some methods called which are trying to violate the contract of the object. This may happen when a thread is trying to modify the Collection object while it is being iterated by some fail-fast iterator, the iterator will throw the exception.
if while iterating over the collection, we directly try to modify that collection, then the given fail-fast iterator will throw this ConcurrentModificationException.
mport java.awt.List;
import java.util.*;
public class Concurrentmodificationexception {
public static void main(String[] args) {
ArrayList<Integer> list = new ArrayList<>();
list.add(1);
list.add(2);
list.add(3);
list.add(4);
list.add(5);
Iterator<Integer> it = list.iterator();
while (it.hasNext()) {
Integer value = it.next();
System.out.println("List Value:" + value);
if (value.equals(3))
list.remove(value); //error
//safe : it.remove(); - list will have now 4 elements
}
}
}
How to avoid ConcurrentModificationException in a single-threaded environment?
By using iterator’s remove() function, you can remove an object from an underlying collection object.
Detected by the JVM (Java Virtual Machine) at runtime.
Handling
Must be fixed by modifying the source code and recompiling.
Can be handled using try-catch blocks or other mechanisms.
Impact on Program
Prevents the program from being compiled successfully.
May cause the program to terminate prematurely.
//example 1
public class CompileTimeErrorExample {
public static void main(String[] args) {
int x = "Hello"; // This line will cause a compile-time error
System.out.println(x);
}
}
//example 2
public class RuntimeErrorExample {
public static void main(String[] args) {
int[] array = new int[3];
System.out.println(array[4]); // This line will cause a runtime error
}
}
Checked vs unchecked exception
Feature
Checked Exception
Unchecked Exception
Definition
Subclasses of java.lang.Exception excluding RuntimeException and its subclasses.
Subclasses of RuntimeException and Error.
Mandatory Handling
Must be handled or declared in the method signature.
Not required to be handled explicitly.
Compile-Time Behavior
Compiler checks for handling or declaration.
Compiler does not enforce handling or declaration.
Intended to handle expected exceptional conditions that a program can reasonably recover from.
Typically indicates programming errors or conditions beyond the control of the programmer.
Checked by Compiler
Yes
No
Need Try-Catch or Throws
Must be caught or declared in the method signature.
Optional, can be caught but not required.
Forced Exception Handling
Yes
No
//Example1
public class CheckedExceptionExample {
public static void main(String[] args) {
try {
FileReader file = new FileReader("file.txt");
// File processing code
} catch (IOException e) {
// Handle the exception
e.printStackTrace();
}
}
}
//example2:
public class UncheckedExceptionExample {
public static void main(String[] args) {
String str = null;
System.out.println(str.length()); // This line will throw NullPointerException
}
}
Checked vs Compile-time exception
Note: Eclipse IDE, as well as many other Integrated Development Environments (IDEs), often perform static code analysis in real-time, providing feedback on potential issues before compilation. This is different from compile-time exceptions, which occur during the actual compilation process.
Feature
Compile-Time Exception
Checked Exception
Definition
Any exception that occurs during compilation due to syntactical or structural issues in the code.
Subclasses of java.lang.Exception excluding RuntimeException and its subclasses.
Mandatory Handling
Not required to be handled explicitly; resolved by fixing the issues identified by the compiler.
Must be handled or declared in the method signature.
Example
Syntax errors, type mismatches, unresolved symbols.
java public class CompileTimeExceptionExample { public static void main(String[] args) { int x = "Hello"; // This line will cause a compile-time error System.out.println(x); } }
java import java.io.FileReader; import java.io.IOException; public class CheckedExceptionExample { public static void main(String[] args) { try { FileReader file = new FileReader("file.txt"); // File processing code } catch (IOException e) { e.printStackTrace(); } } }
unchecked vs runtime exception
Feature
Unchecked Exception
Runtime Exception
Definition
Subclasses of Error.
Subclasses of RuntimeException and Error.
Mandatory Handling
Not required to be handled explicitly.
Not required to be handled explicitly.
Example Code Snippet
public class StackOverflowErrorExample { public static void recursiveMethod(int n) { if (n == 0) { return; // Base case to prevent infinite recursion } recursiveMethod(n); // Recursive call without a decrementing argument } public static void main(String[] args) { recursiveMethod(100000); // Large number of recursive calls } } //Note:The default stack size can vary depending on your operating system and JVM configuration. You can sometimes adjust it using flags like -Xss when starting the JVM process.
String str = "abc"; int num = Integer.parseInt(str); // Throws NumberFormatException
In-built classes:
Subclasses of Error: StackOverflowError OutOfMemoryError VirtualMachineError InternalError
Definition- An Immutable Object means that the state of the Object cannot change after its creation. Changing an object means to use its methods to change one of its fields or the fields are public (and not final ), so that you can be updated from outside without accessing them via methods.
Example Of Immutable class:
String
Primary Wrapper class
String – The String is immutable means that you cannot change the object itself, but you can change the reference to the object.while the String object is immutable, its reference variable is not.
Advantages of immutability
Thread safety : Immutability of strings has positive significance in multithreaded applications because it provides automatic thread safety. Instances of immutable types are inherently thread-safe , because no thread can modify it.
Security reasons- String is widely used as parameter for many java classes, e.g. network connection, opening files, etc. Were String not immutable , a connection or file would be changed and lead to serious security threat.
ClassLoader:– A ClassLoader in Java uses a String object as an argument. Consider, if the String object is modifiable, the value might be changed and the class that is supposed to be loaded might be different.To avoid this kind of misinterpretation, String is immutable
Heap Space – The immutability of String helps to minimize the usage in the heap memory. When we try to declare a new String object, the JVM checks whether the value already exists in the String pool or not. If it exists, the same value is assigned to the new object. This feature allows Java to use the heap space efficiently. (verify this using commoands!)
Drawback
They require more resources than other object types. Because each time it modified, it creates a new one.
Solution – you should use the Java StringBuilder class, which allows you to modify the text without making new String class instances. Java StringBuilder is a Mutable type, that means when you alter it , it still holding same data
Why String class is Final in Java?
The reason behind the String class being final is because no one can override the methods of the String class. So that it can provide the same features to the new String objects as well as to the old ones.
Why Primitive Wrapper Classes are Immutable in Java? -Same reason as the advantage if immutability on object
How to create immutable class in java?
5 steps –
Theclass must be declared as final so that child classes can’t be created.
Data members in the class must be declared private so that direct access is not allowed.
Data members in the class must be declared as final so that we can’t change the value of it after object creation.
A parameterized constructor should initialize all the fields performing a deep copy so that data members can’t be modified with an object reference.
You need to take special care while working with mutable object. You need to do deep copy in case of imutable objects.
Deep Copy of objects should be performed in the getter methods to return a copy rather than returning the actual object reference)
If you return clone of object from getter method, it won’t return original object, so your original object will remain intact.
Example
Immutability implementation With bugs
public final class Country{
private final String countryName;
private final ArrayList listOfStates;
public Country(String countryName,ArrayList listOfStates) {
super();
this.countryName = countryName;
this.listOfStates=listOfStates;
}
public String getCountryName() {
return countryName;
}
public ArrayList getListOfStates() {
return listOfStates;
}
public static void main(String args[])
{
ArrayList listOfStates=new ArrayList();
listOfStates.add("Madhya Pradesh");
listOfStates.add("Maharastra");
listOfStates.add("Gujrat");
Country country=new Country("India",listOfStates);
System.out.println("Country : "+country.getCountryName());
System.out.println("List of states : "+country.getListOfStates());
// It will be added to the list because we did not use clone in getListOfStates
country.getListOfStates().add("Kerala");
// It will be added to the list because we did not use deep copy in constructor
listOfStates.add("Rajasthan");
System.out.println("Updated List of states : "+country.getListOfStates());
}
}
Above class is not immutable. There are two reasons for it :
We did not use clone in getListOfStates() method, so we are able to add “Kerala” to the listOfStates.
We did not do deep copy for listOfStates , so we are able to add “Rajasthan” to the list.
Lets use clone in getListOfStates() method and see the difference, just change getListOfStates() to below code:
public ArrayList getListOfStates() {
return (ArrayList) listOfStates.clone();
}
//in case of mutable map
//in getter-create temp map and assign valuer iof instance member and return temp
Outpu-
Country : India
List of states : [Madhya Pradesh, Maharastra, Gujrat]
Updated List of states : [Madhya Pradesh, Maharastra, Gujrat, Rajasthan]
If you notice, “Kerala” is not added to the list because we are returning clone of listOfStates in getListOfStates() method, so adding “Kerala” to country.getListOfStates() won’t affect original list. We are one step closed to immutable class now. Lets change constructor to make deep copy of listOfStates object
public Country(String countryName, ArrayList listOfStates) {
super();
this.countryName = countryName;
ArrayList tempList = new ArrayList();
for (int i = 0; i < listOfStates.size(); i++) {
tempList.add(listOfStates.get(i));
}
this.listOfStates = tempList;
}
Concurrent vs parallelism
Number of threads required
Why wait/notify/notifyall() is in the Object Clas not Thread
memory is allocated only once at the time of class loading.while execution of a static method the whole class is blocked. So other static synchronized methods are also blocked.
memory is allocated multiple time whenever method is called.
Monitor scope[meaning of monitor]
the monitor belongs to the class
the monitor belongs to the instance.
Thread Future use .. apart from type .. where to use where not to use.
CV- rEQUES -aSYNC Queue .. if inflows if heavy and queue is full how to deal with it?
ThreadLocal
The TheadLocal construct allows us to store data that will be accessible only by a specific thread.
Let’s say that we want to have an Integer value that will be bundled with the specific thread:
ThreadLocal<Integer> threadLocalValue = new ThreadLocal<>();
Next, when we want to use this value from a thread we only need to call a get() or set() method. Simply put, we can think that ThreadLocal stores data inside of a map – with the thread as the key.
Due to that fact, when we call a get() method on the threadLocalValue, we will get an Integer value for the requesting thread:
threadLocalValue.set(1); Integer result = threadLocalValue.get();
We can construct an instance of the ThreadLocal by using the withInitial() static method and passing a supplier to it:
To remove the value from the ThreadLocal, we can call the remove() method:
threadLocal.remove();
ThreadLocals and Thread Pools
ThreadLocal provides an easy-to-use API to confine some values to each thread. This is a reasonable way of achieving thread-safety in Java. However, we should be extra careful when we’re using ThreadLocals and thread pools together.
In order to better understand the possible caveat, let’s consider the following scenario:
First, the application borrows a thread from the pool.
Then it stores some thread-confined values into the current thread’s ThreadLocal.
Once the current execution finishes, the application returns the borrowed thread to the pool.
After a while, the application borrows the same thread to process another request.
Since the application didn’t perform the necessary cleanups last time, it may re-use the same ThreadLocal data for the new request.
This may cause surprising consequences in highly concurrent applications.
One way to solve this problem is to manually remove each ThreadLocal once we’re done using it. Because this approach needs rigorous code reviews, it can be error-prone.
Extending the ThreadPoolExecutor
As it turns out, it’s possible to extend the ThreadPoolExecutor class and provide a custom hook implementation for beforeExecute() and afterExecute() methods. The thread pool will call the beforeExecute() method before running anything using the borrowed thread. On the other hand, it will call the afterExecute() method after executing our logic.
The optimum size of the thread pool depends on the number of processors available and the nature of the tasks.
On a N processor system for a queue of only computation type processes, a maximum thread pool size of N or N+1 will achieve the maximum efficiency.
But tasks may wait for I/O and in such a case we take into account the ratio of waiting time(W) and service time(S) for a request; resulting in a maximum pool size of N*(1+ W/S) for maximum efficiency.
Youtube-
Programming– Concurrency vs parallelism
Parallel = Two queues and two coffee machines.
Concurrent = Two queues and one coffee machine.
S.NO
Concurrency
Parallelism
1.
Concurrency is the task of running and managing the multiple computations at the same time.
While parallelism is the task of running multiple computations simultaneously.
2.
Concurrency is achieved through the interleaving operation of processes on the central processing unit(CPU) or in other words by the context switching.
While it is achieved by through multiple central processing units(CPUs).
3.
Concurrency can be done by using a single processing unit.
While this can’t be done by using a single processing unit. it needs multiple processing units.
4.
Concurrency increases the amount of work finished at a time.
While it improves the throughput and computational speed of the system.
5.
Concurrency deals lot of things simultaneously.
While it do lot of things simultaneously.
6.
Concurrency is the non-deterministic control flow approach.
While it is deterministic control flow approach.
7.
In concurrency debugging is very hard.
While in this debugging is also hard but simple than concurrency.
Computing – parallel vs Distributed
Parallel Computing: In parallel computing multiple processors performs multiple tasks assigned to them simultaneously. Memory in parallel systems can either be shared or distributed. Parallel computing provides concurrency and saves time and money.
Parallel execution – on multiple CPU
Distributed Computing: In distributed computing we have multiple autonomous computers which seems to the user as single system. In distributed systems there is no shared memory and computers communicate with each other through message passing. In distributed computing a single task is divided among different computers.
Section 6 : Java Special purpose classes
Topics
Cloning
ShallowCopy & DeepCopy
Serialization & Externalization
Comparable & Comparator
StringBuffer & StringBuilder
Hash-code and equals contract
How cloning works?
Definition- cloning is about creating the copy of original object. I
By default, java cloning is ‘field by field copy’ i.e. as the Object class does not have idea about the structure of class on which clone() method will be invoked
So, JVM when called for cloning, do following things:
If the class has only primitive data type members then a completely new copy of the object will be created and the reference to the new object copy will be returned.
If the class contains members of any class type then only the object references to those members are copied and hence the member references in both the-original-object as well as the cloned-object refer to the same object.
Apart from above default behaviour, you can always override this behavior and specify your own. This is done using overriding clone() method. Let’s see how it is done.
Java Cloneable interface and clone() method
If a class needs to support cloning it has to do following things:
You must implement Cloneable interface.
You must override clone() method from Object class. [Its weird. clone() method should have been in Cloneable interface.]
Java docs about clone() method are given below (formatted and extract).
//mention the three axioms of clone.
/*Java clone() method
Creates and returns a copy of this object. The precise meaning of "copy" may depend on the class of the object.
The general intent is that, for any object x, the expression:
1) x.clone() != x will be true .
2) x.clone().getClass() == x.getClass() will be true, but these are not absolute requirements.
3) x.clone().equals(x) will be true, this is not an absolute requirement.
*/
protected native Object clone() throws CloneNotSupportedException;
First statement guarantees that cloned object will have separate memory address assignment
Second statement suggest that original and cloned objects should have same class type, but it is not mandatory.
Third statement suggest that original and cloned objects should have be equal using equals() method, but it is not mandatory
Let’s understand Java clone with example. Our first class is Employee class with 3 attributes – id, name and department.
TestCloning.java
public class TestCloning
{
public static void main(String[] args) throws CloneNotSupportedException
{
Department dept = new Department(1, "Human Resource");
Employee original = new Employee(1, "Admin", dept);
//Lets create a clone of original object
Employee cloned = (Employee) original.clone();
//Let verify using employee id, if cloning actually workded
System.out.println(cloned.getEmpoyeeId());
//Verify JDK's rules
//Must be true and objects must have different memory addresses
System.out.println(original != cloned);
//As we are returning same class; so it should be true
System.out.println(original.getClass() == cloned.getClass());
//Default equals method checks for references so it should be false. If we want to make it true,
//then we need to override equals method in Employee class.
System.out.println(original.equals(cloned));
}
}
//Output:
//1
//true
//true
//false
Employee.java
public class Employee implements Cloneable{
private int empoyeeId;
private String employeeName;
private Department department;
public Employee(int id, String name, Department dept)
{
this.empoyeeId = id;
this.employeeName = name;
this.department = dept;
}
@Override
protected Object clone() throws CloneNotSupportedException {
return super.clone();
}
//Getters and Setters
}
Department.java
public class Department
{
private int id;
private String name;
public Department(int id, String name)
{
this.id = id;
this.name = name;
}
//Getters and Setters
}
Great, we successfully cloned the Employee object. But, remember we have two references to the same object and now both will change the state of the object in different parts of the application. Want to see how? Let’s see.
Original and cloned - both changes
public class TestCloning {
public static void main(String[] args) throws CloneNotSupportedException {
Department hr = new Department(1, "Human Resource");
Employee original = new Employee(1, "Admin", hr);
Employee cloned = (Employee) original.clone();
//Let change the department name in cloned object and we will verify in original object
cloned.getDepartment().setName("Finance");
System.out.println(original.getDepartment().getName());
System.out.println(cloned.getDepartment().getName());
}
}
//Output:
//Finance
//Finance
Oops, cloned object changes are visible in original also. This way cloned objects can make havoc in the system if allowed to do so. Anybody can come and clone your application objects and do whatever he likes. Can we prevent this??
Answer is yes, we can. We can prevent this by creating Java deep copy and use copy constructors. We will learn about them later in this post. Let’s first see what is deep cloning and shallow cloning in Java.
Java Shallow Copy
Shallow clone is “default implementation” in Java. In overridden clone method, if you are not cloning all the object types (not primitives), then you are making a shallow copy.
All above examples are of shallow copy only, because we have not cloned the Department object on Employee class’s clone method. Now, I will move on to next section where we will see the deep cloning.
Java Deep Copy
In the deep copy, we create a clone which is independent of original object and making changes in the cloned object should not affect original object.
Let see how deep copy is created in Java.(additional copying is needed part from overriding clonew() if class has refernce to other Object)
I modified the Employee classes clone() method and added following clone method in Department class.
//Department.java
//Defined clone method in Department class.
@Override
protected Object clone() throws CloneNotSupportedException {
return super.clone();
}
Now testing our cloning code gives desired result and name of the department will not be modified.
public class TestCloning
{
public static void main(String[] args) throws CloneNotSupportedException
{
Department hr = new Department(1, "Human Resource");
Employee original = new Employee(1, "Admin", hr);
Employee cloned = (Employee) original.clone();
//Let change the department name in cloned object and we will verify in original object
cloned.getDepartment().setName("Finance");
System.out.println(original.getDepartment().getName());
System.out.println(cloned.getDepartment().getName());
}
}
//Output:
//Human Resource
//Finance
So deep cloning requires satisfaction of following rules –
All the member classes in original class should support cloning and in clone method of original class in context should call super.clone() on all member classes.
If any member class does not support cloning then in clone method, one must create a new instance of that member class and copy all its attributes one by one to new member class object. This new member class object will be set in cloned object.
Java Copy Constructors -way to clone without using Cloneable Interface and clone() method
Copy constructors are special constructors in a class which takes argument for its own class type. So, when you pass an instance of class to copy constructor, then constructor will return a new instance of class with values copied from argument instance. It helps you to clone object without Cloneable interface.
Comparable provides compareTo() method to sort elements in Java.
Comparator provides compare() method to sort elements in Java.
Comparable interface is present in java.lang package.
Comparator interface is present in java.util package.
The logic of sorting must be in the same class whose object you are going to sort.
The logic of sorting should be in separate class to write different sorting based on different attributes of objects.
The class whose objects you want to sort must implement comparable interface.
Class, whose objects you want to sort, do not need to implement a comparator interface.
It provides single sorting sequences. [since the compareTo(T obj) method is in same class]
It provides multiple sorting sequences.[since the compare(T obj,T obj1) method can be used with multiple class to provide multiple sorting sequence]
This method can sort the data according to the natural sorting order.
This method sorts the data according to the customised sorting order.
It affects the original class. i.e., actual class is altered.
It doesn’t affect the original class, i.e., actual class is not altered.
Implemented frequently in the APIby: Calendar, Wrapper classes, Date, and String.
It is implemented to sort instances of third-party classes.
NullPointerException– if the specified object is null ClassCastException – if the specified object’s type prevents it from being compared to this object.
We can sort the list elements of Comparable type by Collections.sort(List) method.
We can sort the list elements of Comparator type by Collections.sort(List, Comparator) method.
Comparable Example:
/**
* @author arsenal Application Runner class
*/
public class ComparingObjectsDemo {
public static void main(String[] args) {
List<Integer> numbsers = Arrays.asList(32, 12, 4, 5, 232, 5, 6, 786, 43, 34);
// In-built Objects already implements Comparable so are sorted as per NO
numbsers.forEach(x -> System.out.println(x));//inorder but not sorted
Collections.sort(numbsers);
numbsers.forEach(x -> System.out.println(x));
// Cusomte Object need to implement comparable
List<Employee> emps = new LinkedList<>();
emps.add(new Employee(Date.valueOf("2005-10-10"), "Sevanand", 11));
emps.add(new Employee(Date.valueOf("2006-10-10"), "Nand", 12));
emps.add(new Employee(Date.valueOf("2002-10-10"), "Anand", 13));
emps.add(new Employee(Date.valueOf("2010-10-10"), "Xevanand", 14));
// emps.forEach(x->System.out.println(x)); //Print In-order of insertion not
// sorted
Collections.sort(emps);// sorted by natural Order //For custom objects need to implement Comparable
// emps.forEach(x -> System.out.println(x));
}
}
/**
* @author arsenal
* Custom Object to be sorted must implement Comparable
*/
class Employee implements Comparable<Employee> {
private Date dob;
private String name;
private int empID;
public Employee(Date dob, String name, int empID) {
super();
this.dob = dob;
this.name = name;
this.empID = empID;
}
public Date getDob() {
return dob;
}
public String getName() {
return name;
}
public int getEmpID() {
return empID;
}
@Override
public int compareTo(Employee o) {
return this.empID - o.empID;//can be sorted based on only one parameter since Comparable part of Object itself
// return this.name.compareTo(o.name);
// return this.dob.compareTo(o.dob);
}
@Override
public String toString() {
return "Employee [dob=" + dob + ", name=" + name + ", empID=" + empID + "]";
}
}
Comparator Example
// A class 'Movie' To be sorted
class Movie {
private double rating;
private String name;
private int year;
// Constructor
public Movie(String nm, double rt, int yr)
{
this.name = nm;
this.rating = rt;
this.year = yr;
}
// Getter methods for accessing private data
public double getRating() { return rating; }
public String getName() { return name; }
public int getYear() { return year; }
}
// Class to compare Movies by name
class NameCompare implements Comparator<Movie>
{
public int compare(Movie m1, Movie m2)
{
return m1.getName().compareTo(m2.getName());
}
}
// Class to compare Movies by ratings
class RatingCompare implements Comparator<Movie>
{
public int compare(Movie m1, Movie m2)
{
if (m1.getRating() < m2.getRating()) return -1;
if (m1.getRating() > m2.getRating()) return 1;
else return 0;
}
}
// Driver class
class Main
{
public static void main(String[] args)
{
ArrayList<Movie> list = new ArrayList<Movie>();
list.add(new Movie("Force Awakens", 8.3, 2015));
list.add(new Movie("Star Wars", 8.7, 1977));
list.add(new Movie("Empire Strikes Back", 8.8, 1980));
list.add(new Movie("Return of the Jedi", 8.4, 1983));
// Sort by rating : (1) Create an object of ratingCompare
// (2) Call Collections.sort
// (3) Print Sorted list
System.out.println("Sorted by rating");
RatingCompare ratingCompare = new RatingCompare();
Collections.sort(list, ratingCompare);
for (Movie movie: list)
System.out.println(movie.getRating() + " " +
movie.getName() + " " +
movie.getYear());
// Call overloaded sort method with RatingCompare
// (Same three steps as above)
System.out.println("\nSorted by name");
NameCompare nameCompare = new NameCompare();
Collections.sort(list, nameCompare);
for (Movie movie: list)
System.out.println(movie.getName() + " " +
movie.getRating() + " " +
movie.getYear());
// Uses Comparable to sort by year
System.out.println("\nSorted by year");
Collections.sort(list);
for (Movie movie: list)
System.out.println(movie.getYear() + " " +
movie.getRating() + " " +
movie.getName()+" ");
}
}
Comparator’s abstract, default and static methods:
Method
Signature
Description
Example Usage
compare(T o1, T o2)
int compare(T o1, T o2)
Compares two objects for order. Returns a negative integer, zero, or a positive integer as the first argument is less than, equal to, or greater than the second.
Returns a comparator that considers null values as less than non-null values.
java Comparator<String> nullsFirstComparator = Comparator.nullsFirst(Comparator.naturalOrder()); int comparison = nullsFirstComparator.compare("apple", null);
nullsLast()
default Comparator<T> nullsLast()
Returns a comparator that considers null values as greater than non-null values.
java Comparator<String> nullsLastComparator = Comparator.nullsLast(Comparator.naturalOrder()); int comparison = nullsLastComparator.compare(null, "banana");
naturalOrder()
static <T extends Comparable<? super T>> Comparator<T> naturalOrder()
Returns a comparator that compares Comparable objects in natural order.
java Comparator<String> naturalOrderComparator = Comparator.naturalOrder(); int comparison = naturalOrderComparator.compare("apple", "banana");
reverseOrder()
static <T> Comparator<T> reverseOrder()
Returns a comparator that reverses the order of the natural ordering of Comparable objects.
java Comparator<String> reverseOrderComparator = Comparator.reverseOrder(); int comparison = reverseOrderComparator.compare("banana", "apple");
String Buffer(thread safe) vs String Builder (not safe but better performance)
No.
String-buffer
StringBuilder
1)
StringBuffer is synchronized i.e. thread safe. It means two threads can’t call the methods of StringBuffer simultaneously.
StringBuilder is non-synchronized i.e. not thread safe. It means two threads can call the methods of StringBuilder simultaneously.
2)
StringBuffer is less efficient than StringBuilder.
StringBuilder is more efficient than StringBuffer.
3)
StringBuffer was introduced in Java 1.0
StringBuilder was introduced in Java 1.5
Hash-code and Equals Contract
In Java, the hashCode and equals methods are part of the Object class, and they play a crucial role in hash-based collections like HashMap, HashSet, etc.
hashCode Method:
According to the contract, if two objects are equal (according to the equals method), they must have the same hash code. However, the reverse is not necessarily true: two objects with the same hash code are not required to be equal.
It is important to override the hashCode method when you override the equals method to maintain this contract.
equals Method:
According to the contract, the equals method must be reflexive, symmetric, transitive, and consistent.
Reflexive: x.equals(x) must return true.
Symmetric: If x.equals(y) returns true, then y.equals(x) must also return true.
Transitive: If x.equals(y) returns true and y.equals(z) returns true, then x.equals(z) must also return true.
Consistent: Repeated calls to equals should consistently return true or false, assuming the objects being compared have not been modified.
It is essential to override the equals method when you want to define custom equality for your objects.
//Format - synatx
@Override
public int hashCode() {
// Calculate and return the hash code for the object
}
//equals
@Override
public boolean equals(Object obj) {
// Compare this object with another object for equality
}
Additional Information
Why 2 object equality should result in hashcode equality?
If Hashcode is not equal for equal object then hash based DS will not work efficiently
Since it uses has-code as key to bucket- since hash-code are not same then same content will point to 2 different buckets
Why 2 Object with same hashcode are not required to be equal
in-efficient implementation of hash-code result in hash-collision and unequal object points to same bucket
//object not same but hashcode same - faulty implemention of haschcode
import java.util.Objects;
class CustomObject {
private String data;
public CustomObject(String data) {
this.data = data;
}
@Override
public int hashCode() {
// Generate the same hash code for all objects
return 42;
}
@Override
public boolean equals(Object obj) {
if (this == obj) return true;
if (obj == null || getClass() != obj.getClass()) return false;
CustomObject that = (CustomObject) obj;
return Objects.equals(data, that.data);
}
}
public class SameHashCodeDifferentEquals {
public static void main(String[] args) {
// Creating two different objects with the same hash code
CustomObject obj1 = new CustomObject("Object 1");
CustomObject obj2 = new CustomObject("Object 2");
// Printing hash codes
System.out.println("Hash code for obj1: " + obj1.hashCode());
System.out.println("Hash code for obj2: " + obj2.hashCode());
// Checking equality
System.out.println("Are obj1 and obj2 equal? " + obj1.equals(obj2));
}
}