Tuesday, March 20, 2012

Capturing Performance of a Command in Linux

Capturing Performance of a Command in Linux

We are interested in checking the time taken to complete a command, is there any way to get to know that in linux.

yes, /usr/bin/time will help to do that.

time command

$/usr/bin/time ls

output will be

a.txt b.txt 0.000u 0.004s 0:00.79 0.0% 0+0k 0+0io 0pf+0w

Hope, this result is confusing, please refe man time . The default format string is

%Uuser %Ssystem %Eelapsed %PCPU (%Xtext+%Ddata %Mmax)k %Iinputs+%Ooutputs (%Fmajor+%Rminor)pagefaults %Wswaps

When the -p option is given the (portable) output format

real %e user %U sys %S

is used.

Format

we can format the result as we wanted

    Time

    • %E Elapsed real time (in [hours:]minutes:seconds).
    • %e (Not in tcsh.) Elapsed real time (in seconds).
    • %S Total number of CPU-seconds that the process spent in kernel mode.
    • %U Total number of CPU-seconds that the process spent in user mode.
    • %P Percentage of the CPU that this job got, computed as (%U + %S) / %E.

    Memory

    • %M Maximum resident set size of the process during its lifetime, in Kbytes.
    • %t (Not in tcsh.) Average resident set size of the process, in Kbytes.
    • %K Average total (data+stack+text) memory use of the process, in Kbytes.
    • %D Average size of the process\u2019s unshared data area, in Kbytes.
    • %p (Not in tcsh.) Average size of the process\u2019s unshared stack space, in Kbytes.
    • %X Average size of the process\u2019s shared text space, in Kbytes.
    • %Z (Not in tcsh.) System\u2019s page size, in bytes. This is a per-sys-tem constant, but varies between systems.
    • %F Number of major page faults that occurred while the process was running. These are faults where the page has to be read in from disk.
    • %R Number of minor, or recoverable, page faults. These are faults for pages that are not valid but which have not yet been claimed by other virtual pages. Thus the data in the page is still valid but the system tables must be updated.
    • %W Number of times the process was swapped out of main memory.

    I/O

    • %I Number of file system inputs by the process.
    • %O Number of file system outputs by the process.
    • %r Number of socket messages received by the process.
    • %s Number of socket messages sent by the process.
    • %k Number of signals delivered to the process.
    • %C (Not in tcsh.) Name and command line arguments of the command being timed.
    • %x (Not in tcsh.) Exit status of the command.

Routing result to file

time command has argument option to route the result to a file system

/usr/bin/time -o /tmp/timeresult.txt ls

output will be like below in file /tmp/timeresult.txt

0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 3440maxresident)k
0inputs+0outputs (0major+272minor)pagefaults 0swaps

Route only elapsed time to file

/usr/bin/time -o -f /tmp/timeresult.txt "ls command elapsed time: %e" ls

output in file looks like below, elapsed time calculated in seconds

ls command elapsed time: 0.91

time command output not able to capture in crontab

crontab is the default scheduler to start the process in partiular time and number of reoccurencess.
Above mentioned option to route may not work in crontab, or inconsistently work, due to crontab works with separate shell, and/or if we try to gather result from different shell.

sh -> sh -c -> time ls

at command

This command helps to start the command in particular time and day. Unfortunately, results the same as crontab.

sleep command

This command is old and golden days one, which helps to run every scheduled time.

sleep [<umber>suffix suffix
  • s seconds
  • m minutes
  • h hours
  • d days

conclusion : time with sleep

If we give a try using these commands like below


for i in {0..20}
do
/usr/bin/time -a -o /tmp/timeresult.txt -f "ls command elapsed time: %e" ls
sleep 1h
done

Every hour onece, this taks wake up and run. No new shell get started every wake up call.

Monday, March 19, 2012

Performance tools and whitepapers

I have captured some of the mostly used tools for performance and monitoring JAVA applications and Database.

Tools in market

J2SDK tools

  • Java Heap Analysis Tooljhat
  • command line utility included in the Solaris and Linux jmap
  • Java Virtual Machine Statistics Monitoring Tool jstat
  • Stack Trace jstack
  • JMX-compliant monitoring tooljConsole
  • Monitoring and Management Using JMX JMX agent
  • Java Virtual Machine Monitoring, Troubleshooting, and Profiling Tool jvisualvm
  • Open Source GC Viewer

Whitepapers

Visitors, please share here any other tool come across.

Tuesday, March 13, 2012

Reference

Pakckage java.lang.ref Description

Classes in this package provides reference-object classes, which support a limited degree of interaction with the garbage collector. Specilitate GC run to take less time and maximize the chance of not getting OutOfMemoryException.

There are four kind of references possible in JAVA

  1. Strong reference - which get created when object new instance done, and reachable without any other Reference objects. Object obj=new Object();
  2. SoftReference - Soft reference objects, which are cleared at the discretion of the garbage collector in response to memory demand.
    Created object in step1 is passed to create SoftReference Object and the same retrieved via SoftReference object.
    java.lang.ref.Reference softReference=new java.lang.ref.SoftReference(obj); All soft references to softly-reachable objects are guaranteed to have been cleared before the virtual machine throws an OutOfMemoryError. Otherwise no constraints are placed upon the time at which a soft reference will be cleared or the order in which a set of such references to different objects will be cleared. Virtual machine implementations are, however, encouraged to bias against clearing recently-created or recently-used soft references.
  3. WeakReference - Weak reference objects, which do not prevent their referents from being made finalizable, finalized, and then reclaimed.
    Weak references are most often used to implement canonicalizing mappings.
    java.lang.ref.Reference weakReference=new java.lang.ref.WeakReference(obj);
    java.util.WeakHashMap
  4. PhantomReference - Phantom reference objects, which are enqueued after the collector determines that their referents may otherwise be reclaimed. Phantom references are most often used for scheduling pre-mortem cleanup actions in a more flexible way than is possible with the Java finalization mechanism.

Notification

A program may request to be notified of changes in an object's reachability by registering an appropriate reference object with a reference queue at the time the reference object is created. Some time after the garbage collector determines that the reachability of the referent has changed to the value corresponding to the type of the reference, it will add the reference to the associated queue. At this point, the reference is considered to be enqueued. The program may remove references from a queue either by polling or by blocking until a reference becomes available.

Reachability

Going from strongest to weakest, the different levels of reachability reflect the life cycle of an object. They are operationally defined as follows:
  • An object is strongly reachable if it can be reached by some thread without traversing any reference objects. A newly-created object is strongly reachable by the thread that created it.
  • An object is softly reachable if it is not strongly reachable but can be reached by traversing a soft reference.
  • An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by traversing a weak reference. When the weak references to a weakly-reachable object are cleared, the object becomes eligible for finalization.
  • An object is phantom reachable if it is neither strongly, softly, nor weakly reachable, it has been finalized, and some phantom reference refers to it.
  • Finally, an object is unreachable, and therefore eligible for reclamation, when it is not reachable in any of the above ways.
JavaDoc

Thursday, March 1, 2012

Tea time talk on Software developments

Software Development Style


Software development lifecycle is unpredict to define. Industry operates in its own way to develop the softwares and its release management cycle.

Factors are identified in certain extent and style of approaching the complex problems and arriving at smaller problem.

Considering other risk involved in factors like resources, cost, time, quality and scope. Delivery become success or delayed success.


In the product industries, development milestone defining for years will help to move in the directions.
However, acquisitions, Intellectual Properties, and competitor are factors will impact the lifecyle and change the directions.

For instance, competitor attracting our customer with various usability features, and performance. If we move towards fixed direction, we will deliver the product.
However, no customer will be there to consume(Now or Never). Hence, plan B should be executed to reach. If we do this, we won't deliver any new product, like riding old car having a mechanic.
This is when, management skill applied to balance existing customer and adhere to industry directions.

Development model, approach, methodology, and processs varies based on the nature of the software and its market. World is always define the rules to break in future.

Few of the important development styles


  • Waterfall model
  • Customer Driven Development
  • Bug Driven Development
  • Test Driven Development
  • Feature Driven Development
  • Spiral model
  • Behavior Driven Development
  • Domain-driven design
  • Service-oriented modeling
  • Agile software development

Recent Posts

Unix Commands | List all My Posts

Texts

This blog intended to share the knowledge and contribute to JAVA Community such a way that by providing samples and pointing right documents/webpages. We try to give our knowledege level best and no guarantee can be claimed on truth. Copyright and Terms of Policy refer blogspot.com