Monday 6 July 2015

Designing database

Database design has evolved greatly over the last 10 years. In the past, it used to be the database analyst job to fine-tune the SQL query and database index to ensure performance. Nowadays, developers play a much more crucial role to ensure the scalability of data.

The database design task, which was autonomy, now becomes an exciting task, which requires a lot of creativity. In this short article, let's walk through an example of real life issue to see how database design has changed.

Requirements

In this example, our business requirement is to build a database to store property information for the country. At first, we need to store any property and its landlords. If a property is being leased, the system need to store tenants information as well. The system should also record activities of properties, including buy, sell and renting.

As a typical database system, the user should be able to query properties out by any information like address, owner name, district, age,... The system need to serve data for both real time query and reporting purpose.

Analysis

It is pretty obvious that there are few entities in this domain like landlord, tenant, transaction and property. Landlord and tenant can be further analysed as people that acts different roles. For example, one person can rent out his house and rent another house to live, which mean he can be the landlord of one property and the tenant of another. That leaves us with 3 major entities: person, property and transaction. Person and property entities have many to many relations to each other. Transaction entity links to one property and at least one person.

If we group some common attributes like occupation, district, building, it is possible to introduce some other sub-entities that may help to reduce redundancy in information.

The era of relational database

If you are one of a developer that being trapped in the relational database era, the only viable choice for persistence is relational database. Naturally, each entity should be stored in a table. If there are relationship between 2 entities, they are likely to refer to each other by foreign keys.

With this setup, there is zero redundancy and every piece of information has the single source of truth. Obviously, it is the most efficient way in term of storage.

There may be an issue here as it is not easy to implement text searching. Whether 10 years ago or today, text search has never been supported well by relational databases. SQL language provides some wildcard matching in the language itself but it is still very far from a full text search.

Assume that you have completed the task of defining database schema, the fine tuning task is normally the job of database analysts; they will look into every individual query, view, index to increase the performance as much as possible.

However, if the readers have spent years working on relational database, it is quite easy to see the limit of this approach. A typical query may involve joining several tables. While it works well for low amount of records, the solutions seem less feasible when the number of tables increase or the amount of records in each table increase. All kinds of tweaks like data sharding, scaling vertically or adding index only help to increase performance up to a certain level. No magic can help if we are going to deal with hundreds millions of records or joining more than 10 tables.

Extension of relational database

To solve the issue, developers have tried several techniques that may scale better than a traditional relational database. Here are some of them:

Database explosion

This technique reverses the process of normalizing data in relational database. For example, instead of joining property with the building, district or country table, we can simply copy all the column of the relevant records to main table. As a consequence, duplication and redundancy happen. There is no single source of truth for sub-entities like building, district, country. In exchange, the joining part in SQL query is simplified.

Explosion is an expensive process that may take hours or even days to run. It sacrifices space, freshness of data in order to increase real time query performance.

Adding document database

In this technique, relational database is still the source of truth. However, to provide text search, important fields were extracted and stored in a document database. For example, knowing that users will search for people by age, gender and name, we can create document that contains these information plus the record id and store them to Solr or Elastic Search server.

Real time query to the system will first be answered by searching in document database. The answer, which includes bunch of record ids will later be used by relational database to load records. In this approach, document database acts like an external index system that help to provide text search capability.

Storing the whole data to a noSQL database

The final choice is storing data to an object-oriented or document database. However, this approach may add a lot of complexity for data maintenance.

To visualize, we can store the whole property or person to database. The property object contains its owners objects. In reverse, the owner object may includes several property objects. In this case, it is quite a hassle to maintain to set of related objects if the data change.

For example, if a person purchases a property, we need to go to the property object to update owner information and go to that person object to update property information.

Combining relational database and noSQL database

The limits of existing methods

After scanning through the approaches mentioned above, let try to find the limit for each approach.
  • Relational database normalizes data before storing to avoid duplication and redundancy. However, by optimizing storage, it causes additional effort on retrieving the data. Taking consideration that database is normally limit by querying time, not storage, it doesn't seem to be a good trade off.
  • Explosion reverses the normalizing process but it cannot offer fresh data as explosion normally take a long time to run. Comparing running explosion with storing the whole entity object to an object-oriented database, the latter option may be easier to maintain.
  • Adding document database offers text search feature but I feel that it should reverses the options to improve scalability. Document database is faster for retrieval while relational database is better for describing relationship. Hence, it doesn't make sense to send the record ids from document database back to relational database for retrieving records. What may happen if there are millions of records id to be found. Retrieving those records from noSQL database is typically faster than relational database.
  • As mentioned above, when these entities are inter-linked, there is no easy way to separate them out to store to an object-oriented database. 
Proposing combination of relational and noSQL database to store data

Thinking about these limits, I feel that the best way to store data should be combining both relational and document database. Both of them will act as source of truth, storing what they do best. Here is the explanation of how should we split the data.

We store the data similarly to a traditional relational database but splitting the columns to 2 types:
  • Columns that store id or foreign keys to other entity ids ("property_id", "owner_id",..) or unique fields
  • Columns that store data ("name", "age",...)
After this, we can remove any data column from relational database schema. It is possible to keep some simple fields like "name", "gender" if they help to give us some clues when looking at records. After that, we can store the full entities in a document database. We should try to avoid making cross-references in stored documents.

Explain the approach by example

Let try to visualize the approach by describing how should we implements some simple tasks
  • Storing a new property owned by a user
    • Configure JPA to only store name and id for each main entity like person, property. Ignore data fields or sub-entities like building, district, country.
    • Store the property object to relational database. As the result of earlier step, only id and name of the property are persisted. 
    • Update property object with persisted ids.
    • Store property object to document database.
    • Store owner object to document database.
  • Querying property directly
    • Sending query to document database, retrieving back record.
  • Querying property based on owner information
    • Sending query to relational database to find all property that belong to the owner.
    • Sending query to document database to find these property by ids.
In the above steps, we want to store records to relational database first because of the auto id generation. With this approach, we have a very thin relational database that only capture relationships among entities rather than the entities them selves. 

Summary of the approach

Finally, let summarize the new approach
  • Treating main entities as independent records.
  • Treating sub-entities as complex properties, to be included as part of main entities.
  • Storing id, name and foreign keys of main entities inside relational database. The relational database is serving as a bridge, linking independent objects in noSQL database.
  • Storing main entities with updated ids to noSQL database.
  • Any CRUD operation will require committing to 2 databases at the same time.
Pros:
  • Off-load the storing data task from relational database but let it do what it can do best, stores relationships.
  • Best of both worlds with text search and scalability of noSQL database and relations searching of relational database.
Cons:
  • Maintaining 2 databases.
  • No single source of truth. Any corruption happen in one of the two databases will cause data loss.
  • Code complexity.
Possible alternative
  • Storing data to a graph database that offer text search capability. This is quite promising as well but I have not done any benchmark to prove feasibility.

Conclusion

The solutions is pretty complex but I found it is interesting because the scalability issue is solved at the code level rather than database level. By splitting the data out, we may tackle the root cause of the issue and be able to find some balance between performance and maintenance effort.

The complexity of this implementation is very high but there is no simple implementation for big data.

Thursday 11 June 2015

Can java optimize empty array allocation?

Yesterday came across a simple optimization case, here is the original method

public String[] getSomeArray() {
    if (nothing) {
        return new String[0];
    }
    // normal processing ignored for brevity 
}

at the first sight the allocation looks quite wasteful, and I am tempted to carry out some micro optimization like

private static final String[] EMPTY = new String[0];
public String[] getSomeArray() {
    if (nothing) {
        return EMPTY;
    }
    // normal processing ignored for brevity 
}

However, another developer Pedro ringed the bell, maybe java can JIT away the allocation all together, and this does looks a very reasonable JIT target.

Let's find out!

jmh to rescue

@Benchmark
public void test1() {
    for (int i = 0; i < 10000; i++) {
        get1();
    }
}
public String[] get1() {
    return new String[0];
}

private static final String[] CONST = {};
@Benchmark
public void test2() {
    for (int i = 0; i < 10000; i++) {
        get2();
    }
}
public String[] get2() {
    return CONST;
}

A benchmark run gave following result which showed that the two methods ran pretty much at the same speed, therefore the actual allocation could be indeed optimized away

test$ java -jar target/benchmarks.jar -f 1
# JMH 1.9.3 (released 28 days ago)
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 20 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.sample.MyBenchmark.test1

# Run progress: 0.00% complete, ETA 00:01:20
# Fork: 1 of 1
# Warmup Iteration   1: 3177146862.839 ops/s
# Warmup Iteration   2: 2969126090.532 ops/s
...
# Warmup Iteration  19: 3904120378.974 ops/s
# Warmup Iteration  20: 3368973982.889 ops/s
Iteration   1: 3273016452.646 ops/s
Iteration   2: 3720653112.375 ops/s
...
Iteration  19: 2940755393.888 ops/s
Iteration  20: 3490675218.425 ops/s


Result "test1":
  3150112425.866 ±(99.9%) 346620443.427 ops/s [Average]
  (min, avg, max) = (2526859466.365, 3150112425.866, 3790445537.196), stdev = 399168618.122
  CI (99.9%): [2803491982.439, 3496732869.293] (assumes normal distribution)


# JMH 1.9.3 (released 28 days ago)
# VM invoker: /Library/Java/JavaVirtualMachines/jdk1.8.0_40.jdk/Contents/Home/jre/bin/java
# VM options: <none>
# Warmup: 20 iterations, 1 s each
# Measurement: 20 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.sample.MyBenchmark.test2

# Run progress: 50.00% complete, ETA 00:00:40
# Fork: 1 of 1
# Warmup Iteration   1: 2646209214.510 ops/s
# Warmup Iteration   2: 3014719359.164 ops/s
...
# Warmup Iteration  19: 3639571958.173 ops/s
# Warmup Iteration  20: 3127621392.815 ops/s
Iteration   1: 3464961418.737 ops/s
Iteration   2: 2827541432.787 ops/s
....
Iteration  19: 2888880315.543 ops/s
Iteration  20: 3109114933.979 ops/s


Result "test2":
  3048325924.714 ±(99.9%) 269904767.209 ops/s [Average]
  (min, avg, max) = (2523324876.886, 3048325924.714, 3573386254.596), stdev = 310822731.303
  CI (99.9%): [2778421157.505, 3318230691.923] (assumes normal distribution)


# Run complete. Total time: 00:01:20

Benchmark           Mode  Cnt           Score           Error  Units
MyBenchmark.test1  thrpt   20  3150112425.866 ± 346620443.427  ops/s
MyBenchmark.test2  thrpt   20  3048325924.714 ± 269904767.209  ops/s
test$ 

to be super conservative, let's check the assembly

  0x00000001051ffa99: movabs $0x11e65a2c8,%rbx  ;   {metadata({method} {0x000000011e65a2c8} 'get1' '()[Ljava/lang/String;' in 'org/sample/MyBenchmark')}

  0x00000001051ffaa3: and    $0x7ffff8,%edx
  0x00000001051ffaa9: cmp    $0x0,%edx

  0x0000000109ae1f51: movabs $0x122f3d440,%rbx  ;   {metadata({method} {0x0000000122f3d440} 'get2' '()[Ljava/lang/String;' in 'org/sample/MyBenchmark')}
  0x0000000109ae1f5b: and    $0x7ffff8,%eax
  0x0000000109ae1f61: cmp    $0x0,%eax

Now we see the exact same native codes were generated.

Case closed.

Java does optimize empty array allocation.


Happy Coding!

by Dapeng

Monday 25 May 2015

Rethinking database schema with RDF and Ontology

When I joined the industry 10 years ago, my first project used relational database. After that, my next project also used relational database. And as you may guess, my next next projects also used relational database. This went on for so long that I almost forgot that table is just one format to store data.

I only found myself interested in other kind of databases 4 years ago when my company slowly moved to BigData analysis and knowledge management. Over these years, the exposure I have with RDF and Ontology has given me an urge to re-visit and re-think about the approach and principle to build database.

In the scope of this article, I will solely focus my thought on the role of database schema. Please do not feel worry even if the term RDF and Ontology sound unfamiliar to you, I will try my best to introduce these concepts in this article.

Background

Graph as knowledge database

As everyone know, creating relational database start with defining the database schema. It doesn't matter which database we choose, we still need to define tables, columns and any foreign key before inserting data. Apparently, we need to decide how the data would look like before making it.

However, this is not a suitable approach for us when building knowledge database. Because the system suppose to store future knowledge rather than current knowledge, it is impossible to figure out how the data will look like. Therefore, we turned our focus out of relational database and looked for other solutions.

There are many NoSQL databases that can support schema-less data but most of them does not fit our requirement because we want to stored linked data. Hence, graph database seems to be the most sensational choice for us.

Resource Description Framework

A round of shopping in the market give us an unease feeling as there is no widely adopted standard for query language. If we choose a graph database, we may end up writing vendor specific implementation, which doesn't seem to be a good strategy to start.

In an attempt to find some common standards, we managed to find Resource Description Framework, a W3C specification for modelling of information in web resources. It seems to be the best choice for us because Resource Description Framework comes with a very simple mechanism to describe resource linkage. The only side effect that every resource need to be identified by an URI.

For example, to describe that Apple produce iPhone 5, which being sold at 600 USD, we need to generate 2 triples like below.

<http://example.org/Apple> <http://example.org/produce> <http://example.org/iPhone5>
<http://example.org/iPhone5> <http://example.org/price> '600 USD'@en

We have no interest in using URI as resource identifier but still need to do so in order to comply to standard. However, we cannot blame RDF because it was invented for the web. The author has a nice dream where we can follow the URI of resource to retrieve it. Obviously, this idea did not come true.

Leave the original idea of RDF one side, the major benefit for us is the query language SPARQL. For example, to figure out the price of any Apple phones, the query should be:

select ?phone ?price where {
       <http://example.org/Apple> <http://example.org/produce> ?phone .
       ?phone <http://example.org/price> ?price
}

However, RDF is just a concept. In order to use RDF and SPARQL, we still need to find a decent API or implementation, preferably in Java. This leaded us to 2 popular choices, Sesame and Apache Jena. Both APIs are widely accepted and have several vendor implementations. Even better, some vendors provide implementations for both of them.

Ontology

In the example above, it is easy to see that to make a meaningful query, we need to know how the data look like. Therefore, we still end up should have some kinds of data schema. However, this schema should act more like meta-data rather than data definition. Generally, we do not need the schema to be pre-defined before inserting data.

To address this issue, there are two strategies. People started with defining a common vocabulary for RDF. Because most of the time, we already know the relationship among resources but do not know the resources themselves, the vocabulary should be good enough to help forming SPARQL query. However, due to the wide scope of describing the world, no vocabulary is enough to fully express every domain. Therefore, there are many domain specific vocabularies.

While the first approach only tackles describing possible relationship among resources, the second approach attempts to describe resources as well. For example, to describe the resources in the previous example, we can define a class named smart phone and a class named manufacturer. After this, we can specify that a manufacturer can produce a phone.

<http://example.org/Apple> <http://example.org/type> <http://example.org/Manufacturer>
<http://example.org/iPhone5> <http://example.org/type<http://example.org/Phone>
<http://example.org/Manufacturer> <http://example.org/produce> <http://example.org/Phone>

Those triples above form an Ontology. Compare with vocabulary, we found Ontology as a more descriptive way of describing data schema because it can tell us which kind of relationship is applicable to which kind of resource. Therefore, we will not waste time figuring out whether iPhone 5 produces Apple or Apple produces iPhone 5.

Ontology plus RDF is a good choice to build knowledge database. While the repository can be an RDF store which can take in any triple, we can build another ontology in a separate space to model the knowledge in main repository.

From our point of view, it is better to use ontology to form query rather than data validation because it will help to slightly decouple data and schema. In practice, it is perfectly acceptable to insert data that does not fit Ontology, as long as they does not contradict.

For example, with the Ontology defined earlier, we should allow inserting knowledge like

<http://example.org/Apple> <http://example.org/produce> <http://example.org/iPod>
<http://example.org/Apple> <http://example.org/produce> <http://example.org/iPad>

but we can reject any knowledge like

<http://example.org/iPhone5> <http://example.org/produce> <http://example.org/Apple>

With this approach, we can allow importing data first, then modelling them later.

Database Schema

A refreshing thought

Relating our approach on building knowledge database with relational database make me feel that the requirement of defining data schema before data insertion is driven by implementation. If we can temporarily forget practical concerns like data indexing, it is possible to insert data first and define data schema later.

This also brought me another thought of whether it is feasible to have more than one data schema for the same piece of data. The idea may sound awkward at first but quite realistic if we look at the daunting task of modelling the world. For example, if we think of Obama as the US president, we may care about the when he started serving, when will he leave office; but if we think of Obama as any other US citizen, then we care more about date of birth, residential area, security number,... In this way, the schema is serving as a perspective for us to inspect and modify resource.

So, if I can travel back to the time people discussing a common query language for database, I would suggest adding some features to SQL to enrich it, or to introduce a new kind of query language that is less strict:

  • Allow insertion without table definition. Automatically create table definition following insertion command parameters.
  • Make the id of record unique per database rather than unique per table. A record can appear in many tables with the same id. An insertion command need to specify which field is the id for the record.
  • The data definition is more generic without size constrain (varchar and int instead of varchar(255) or int(11)).
  • The select command must comply to the table definition. 
  • It is possible to share a field between two tables for the same record.
Before wrapping up this article, let try to do an quick exercise of building an database that can implements these extended features. The underlying storing system can be any schema-less engine but we can use RDF store for simplicity.

Requirements
  • Insert Obama into US citizen table with name and age and gender. The identifier field is name.
  • Insert Obama into US president table with name, age and elected year. The identifier field is name.
  • Define US citizen table with field name and age.
  • Define US president table with name, age and elected year.
  • Select record from US citizen table will only show name and age as gender is not defined in table definition.
  • Update Obama record in US President table with new age will affect the age in US citizen table because it is a sharing field.
Implementations

Step 1
  • SQL: insert into citizen(name, gender, age) value ('Barack Obama', 'Male', 53)
  • Triples:
    • <Barack Obama> <type> <citizen>
    • <Barack Obama> <name> 'Barrack Obama'
    • <Barack Obama> <gender> 'Male'
    • <Barack Obama> <age> 53
Step 2
  • SQL: insert into president(name, elected_year) value ('Barrack Obama', 'Male', 53)
  • Triples:
    • <Barack Obama> <type> <president>
    • <Barack Obama> <elected_year> 2009
Step 3
  • SQL: create table citizen ('name' varchar, 'age' int, primary key ('name') )
  • Triples:
    • <citizen> <field> <name>
    • <citizen> <field> <age>
    • <citizen> <primary_key> <name>
Step 4
  • SQL: create table president ('name' varchar, 'elected_year' int, primary key ('name') )
  • Triples:
    • <president> <field> <name>
    • <president> <field> <elect_year>
    • <president> <primary_key> <name>
Step 5
  • SQL: select * from citizen
  • SPARQL: select ?record ?field_name ?field_value where {
    • ?record <type> <citizen>
    • <citizen> <field> ?field_name
    • ?record ?field_name ?field_value
  • }
 Step 6
  • SQL: update president set age=54 where name='Barack Obama'
  • Overwrite <Barack Obama> <age> 53 with <Barack Obama> <age> 54

Conclusions

I think some of the ideas above a bit hard to digest if audiences are not familiar with RDF or Ontology. Still, I hope it can raise some more thoughts on bridging the gap between relational database and knowledge database.

Tuesday 12 May 2015

First Agile impression

Last year, we had a mass recruitment for Java developers with various level of experience. Unfortunately, from this part of the world (Asia) Agile has not been very widely adopted. Therefore, we ended up spend extra effort getting new members to be familiar with Agile and XP.

Over the team forming process, which lasted about 6 months, most of the new team members provided positive feedback about working in an Agile environment. In this article, I would like to share with audiences one of the feedback that we have. Enjoy!

It has been a quickfire six months working in an agile environment. This is my first foray into agile software development in nearly a  decade of working in software. Hearing and reading about it could not have prepared me for the actual experience of working in this peculiar environment. However, the overwhelming sentiment for me is - what a breath of fresh air!

The morning standups were unsettling, to put it mildly. Suddenly, you stopped having excuses for not having done anything the day before :). They take some getting used to, but slowly I’m learning that it is more a sharing session than a status update. Easier said than done, but I’m getting there.  Not sure what you're going to do today? Tell everyone - there’s always something to do.

I like the inherent openness that Agile brings to the table. Seemingly mundane things like outstanding tasks become more explicit and we are all the better for it. Finish a task at hand? No one's stopping you from going to the board and picking a new one. Stuck with a sticky issue? Bring it up in the standup and more often than not, offers of help can be expected. Oftentimes though, a quick holler is all that will be required.

Whole days dedicated to planning and retrospectives demand concentration and focus and more often than not, creativity. I have found that they mean less disruption during the iteration for the most important work of all - writing good code. Worthwhile, no?

We practice pair programming. For someone new to this, the intensity is unexpected as you try to  align yourself to your pair's thought process in a continuous back-and-forth cycle. Overall, I have found it to be quite draining, but I believe the upside cannot be underestimated. Each pairing session, even with the same colleague, seems to involve a whole new dynamic and the constant adjustment needed can be likened to a skill. Working in such close proximity can be a double-edged sword, though. Some friction is inevitable and I have often experienced a whole range of emotions whilst working through a pairing session. I try to manage these emotions and reflect upon them afterwards to understand why I’d felt the way I’d felt and how I could have done better. It is undeniably rewarding though - I have learnt a great deal about myself and my pairs during those sessions.

Test-driven development is standard practice here. Mastering, or rather, adhering to the red-green-refactor pattern seemed counter-intuitive at first, but it starts to make sense after a while. In my limited experience, writing tests becomes more fluid with practice. Since there is almost never an excuse for not providing test coverage for any code that will see the light of day (much less production code!), buckling down and writing that test will be a good habit to develop and one which I am convinced will prove to be an invaluable skill and an integral part in my journey to become a better developer.

Building software has never been quite so engaging and dare I say it..fun. Having a close knit team definitely helps. Hopefully, we can keep the good momentum and spirit going as we welcome new members into our fold and the workload ramps up. It will not be easy, but nobody said it would be. Whatever the future holds, I await with bated breath. Onwards.


Zhi Liang

Saturday 18 April 2015

Migrating Spring Web MVC from JSP to AngularJS

Target Audience

This article is written for Spring Web MVC developers who are familiar with JSP development and who would like to understand how to migrate to a client side Javascript framework like AngularJS.

Sample Pet Clinic for reference

An example of a Spring Pet clinic application that we have tried to revamp as an AngularJS app with an updated design can be found here . You can refer to this project on how we introduced AngularJS to the project.

Introduction to AngularJS

AngularJS is a Javascript framework created at Google that touts itself as a "Superheroic Web MVW Framework" (where the "W" in the "MVW" being a tongue-in-cheek reference to "Whatever" for all the various MVx architecture. As it is based on an MVx architecture, AngularJS provides a structure to Javascript development and thus gives Javascript an elevated status compared to traditional Spring + JSP application that probably uses Javascript to provide that bit of interactivity on the user interface. With AngularJS, your Javascript application will also inherit features like Dependency-Injection, HTML-vocabulary extension (via the use of custom directives), unit-testing and functional testing integration as well as DOM-selectors ala JQuery (using jqLite as it provides only a subset of JQuery but you could also easily use JQuery if you prefer). AngularJS also introduces scopes to your Javascript code so that variables declared in your code are bound only to the scope that is required.This prevents variables pollution that inadvertently arises when the size of your Javascript grows. When you are developing a Spring Web MVC application using JSP, you will likely use the Spring-provided form tags to bind your form inputs to a server side model. Similarly, AngularJS provides a way to bind form inputs to models on the client side. In fact, it provides instantaneous 2-way data-binding from the form input to your model on the Javascript application. That means that not only do you have the benefits of having your view updated with changes inside your Javascript model, any changes you make to your UI will also update the Javascript model (and consequently any other views that is bound to that model). It is almost magical to see all the views that are bound to the same JS model on the app update itself automatically. Moreover, since your model can be set to a particular scope, only views that belong to the same scope will be affected, allowing you to sandbox code that should be local only to a particular portion of your view. (This is done via an AngularJS attribute called ng-controller that is set in your HTML templates). You can see the difference in a later section comparing JSP tags and AngularJS directives.

You can see an illustration of the difference here

You can also see this in action with the following embedded plunker code below. Using AngularJS, it is possible to write relatively complex User Interfaces in an organised and elegant manner, always encapsulating the required logic within your components and never running the risk of errant global Javascript variables polluting your scope. It is also very testable, and there are built-in mechanisms to perform tests at the unit and functional level, ensuring that all your User Interface codebase goes through the same rigourous testing that your Java/Spring code undergoes, ensuring quality even at your user interfaces. Another advantage of using AngularJS to write your html templates is that the templates are essentially similar to html even with the various frontend logic baked into your view. It is possible to incorporate AngularJS logic into your template and still pass a HTML validation. Try viewing a JSP file from a browser with all the template logic in place and most likely your browser will give up rendering the page. You can see how a typical AngularJS template looks like :
<div class="row thumbnail-wrapper">
  <div data-ng-repeat="pet in currentOwner.pets" class="col-md-3">
    <div class="thumbnail">
      <img data-ng-src="images/pets/pet{{pet.id % 10 + 1}}.jpg" 
        class="img-circle" alt="My Pet Image">
      <div class="caption">
        <h3 class="caption-heading" data-ng-bind="pet.name">Basil</h3>
        <p class="caption-meta" data-ng-bind="pet.birthdate">08 August 2012</p>
        <p class="caption-meta"><span class="caption-label" 
           data-ng-bind="pet.type.name">Hamster</span></p>
      </div>
      <div class="action-bar">
        <a class="btn btn-default" data-toggle="modal" data-target="#petModal" 
          data-ng-click="editPet(pet.id)">
          <span class="glyphicon glyphicon-edit"></span> Edit Pet
        </a>
        <a class="btn btn-default">
          <span></span> Add Visit
        </a>
      </div>
    </div>
  </div>
</div>
You can probably spot some non-HTML addition to the template includes attributes like
data-ng-click which maps a click on a button to a method name call. There's also data-ng-repeat which will loop through a JSON array and generate the necessary html to render the same view for each item in the array. Yet with all these logic in place, we are still able to validate and view the html template from the browser. AngularJS calls all the non-html tags and attributes "directives" and the purpose of these directives is to enhance the capabilities of HTML. AngularJS also supports both HTML 4 and 5 so if you have templates that are still relying on HTML 4 DOCTYPEs, it should still work fine although the validators for HTML 4 will not recognize data-ng-x attributes.

Scopes in AngularJS

One important concept to grasp in AngularJS is that of scopes. In the past, whenever I had to write javascript for my web application, I had to manage the variable names and construct special name-spaced objects in order to store my scoped properties. However, AngularJS does it for you automatically based on its MVx concept. Every directive will inherit a scope from its controller (or if you would like, an isolate scope that does not inherit scope properties) where the properties and variables created in this scope does not pollute the rest of the scopes or global context. Scopes are used as the "glue" of an AngularJS application. Controllers in AngularJS use scopes to interact with the views. Scopes are also used to pass models and properties between directives and controllers. The advantage of this is that we are now forced to design our application in a way which components are self-contained and relationships between components have to be considered carefully through a use of a model that can be prototypically inherited from a parent scope. A scope can be nested in another scope prototypically in the same way Javascript implements its inheritance model via prototyping. However, one must be careful that any property name that is declared in the child scope that is similar to the parent will be hide the parent property from the child scope thereafter. An example of this can be described in the code below




At the very top in the hierarchy of scopes is the $rootScope, a scope that is accessible globally and can be used as the last resort to share properties and models across the whole application. The use of this should be minimized as it introduces a sort of "global" variable that can pose the same problems when it is over-used.

More information about scopes can be gleaned from the AngularJS documentation found here .

Directives in AngularJS

Directives is one of the most important concept in AngularJS and it is indeed the foundation of what we constitute AngularJS in the markup. It is essentially all the additional customized markup in the form of element, attributes, classes or comments in our HTML markup that gives the markup new functionalities. Consider the following code snippet that demonstrates a customized directive called wdsCustom that will replace the markup element <wds-custom company="wds"> with markup that contains information about a model called "wds" that is declared in the controller scope that wraps the directive. You can have a look at the files app.js, index.html and directive template wds-custom-directive.html to see how this works in the below plunkr snippet. As this article does not attempt to teach you how to write a directive, you can refer to the official documentation here.

Differences in architecture between JSP and AngularJS

When one migrates from a server-side templating engine like JSP or Thymeleaf to a Javascript-based templating engine, one has to experience a paradigm shift towards a client-server architecture. One has to cease thinking of the view as being a part of the web application and instead conceive the web application as 2 separate client-side and server-side applications. An illustration of this is in the next diagram which shows how a Spring application becomes a provider of RESTful Web Services, servicing various front end applications including an AngularJS browser-based application as well as a possibility to provide services for mobile clients like tablets or smartphones. These services could include OAuth, Authentication and other business logic services which should be obfuscated from public view. One should bear in mind that any data or business logic that is published in the form of JSON or javascript files are exposed for the client-side to see. Thus, if there's any business sensitive logic or workflow that should not be exposed, these logic should only be performed on the backend.

Also, when you are working in AngularJS, we prefer to encapsulate form submissions in a JSON object which is sent over to the backend RESTful service via a AngularJS HTTP Post method call (instead of using the HTML form submission). All validation can be performed on the front end using AngularJS's built-in input validation (or if you like to have your custom validation, it can also be easily written) before posting your data to the server. Of course, it is always prudent to validate the same data at the server side as well to ensure that clients that do not check their data do not compromise the integrity of the data on the server side. 

Application structure

One question when we migrate over to AngularJS are the questions on how we would want to organise our AngularJS + Spring application. At my company WDS, we are using Maven as our dependency and package management tool for Java/Spring and that influenced how we decided to place our AngularJS javascript application. The AngularJS application is created within src/main/webapp and the main files are
components/ # the various components are stored here.
js/app.js   # where we bootstrap the application
plugins/    # additional external plugins e.g. jquery.
services/   # common services are stored here.
images/
videos/
You can see an image capture of the folder structure in Eclipse below.





This way of grouping the resources uses the feature-grouping method. There are also ways to group your resources based on types, e.g. grouping all your controllers, services and views into its namesake folder. There are pros and cons for either option but we prefer to use the feature grouping way to contain our resources.

Comparisons betwen AngularJS and JSP custom tags

If you have used Spring's custom form tags in your JSPs for developing your forms, you may be wondering if AngularJS provides the same kind of convenience for mapping form inputs to objects. The answer is yes! As a matter of fact, it is easy to bind any HTML element to a Javascript object. The only difference is that now the binding occurs on the client-side instead of the server-side.
<form:form method="POST" commandName="user">
<table>
    <tr>
        <td>User Name :</td>
        <td><form:input path="name" /></td>
    </tr>
    <tr>
        <td>Password :</td>
        <td><form:password path="password" /></td>
    </tr>
    <tr>
        <td>Country :</td>
        <td>
            <form:select path="country">
            <form:option value="0" label="Select" />
            <form:options items="${countryList}" itemValue="countryId" itemLabel="countryName" />
            </form:select>
        </td>
    </tr>
</table>
</form:form>
Here is an example of the same form in AngularJS
<form name="UserForm" data-ng-controller="ExampleUserController">
  <table>
    <tr>
        <td>User Name :</td>
        <td><input data-ng-model="user.name" /></td>
    </tr>
    <tr>
        <td>Password :</td>
        <td><input type="password" data-ng-model="user.password" /></td>
    </tr>
    <tr>
        <td>Country :</td>
        <td>
            <select data-ng-model="user.country" data-ng-options="country as country.label for country in countries">
               <option value="">Select<option />
            </select>
        </td>
    </tr>
</table>
</form>
Form inputs in AngularJS are augmented with additional capabilities like the ngRequired directive that makes the field mandatory based on certain conditions. There are also built-in validation for checking ranges, dates, patterns etc.. You can find out more at AngularJS's official documentation found here which provides all the relevant form input directives.

Considerations when moving from JSP to AngularJS

If you are considering migrating from JSP to AngularJS, there are a few factors to consider for a successful migration.

Converting your Spring controllers to RESTful services

You will need to transform your controllers so instead of forwarding the response to a templating engine to render a view to the client, you will provide services that will be serialised into JSON data instead. The following is an example of how a standard Spring MVC controller RequestMapping uses the ModelAndView object to render a view with the Owner as described in the url mapping.
@RequestMapping("/api/owners/{ownerId}")
public ModelAndView showOwner(@PathVariable("ownerId") int ownerId) {
    ModelAndView mav = new ModelAndView("owners/ownerDetails");
    mav.addObject(this.clinicService.findOwnerById(ownerId));
    return mav;
}
A controller RequestMapping like that can be converted into an equivalent RESTful service that returns the owner based on the ownerId. Your template can then be moved into AngularJS which will then bind the owner object to the AngularJS template.
@RequestMapping(value = "/api/owners/{id}", method = RequestMethod.GET)
public @ResponseBody Owner find(@PathVariable Integer id) {
    return this.clinicService.findOwnerById(id);
}
In order for Spring MVC to convert your return object (which needs to be Serializable) to a JSON object, you need to configure your Spring context file to include a conversion service that uses Jackson2 for JSON serialisation. An example of the snippet that performs this Spring context configuration is shown below.
<bean id="objectMapper" class="org.springframework.http.converter.json.Jackson2ObjectMapperFactoryBean" p:indentOutput="true" p:simpleDateFormat="yyyy-MM-dd'T'HH:mm:ss.SSSZ"></bean>
<mvc:annotation-driven conversion-service="conversionService" >
 <mvc:message-converters>
  <bean class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter" >
   <property name="objectMapper" ref="objectMapper" />
  </bean>
 </mvc:message-converters>
</mvc:annotation-driven>
Once you have converted your controllers into RESTful services, you can then access these resources from your AngularJS application. One nice way to access RESTful services in AngularJS is to use the built-in ngResource directive that allows you to access your RESTful services in an elegant and concise manner. An example of the Javascript code to access RESTful services using this directive can be illustrated by the following:
var Owner = ['$resource','context', function($resource, context) {
 return $resource(context + '/api/owners/:id');
}];

app.factory('Owner', Owner);

var OwnerController = ['$scope','$state','Owner',function($scope,$state,Owner) {
 $scope.$on('$viewContentLoaded', function(event){
  $('html, body').animate({
      scrollTop: $("#owners").offset().top
  }, 1000);
 });

 $scope.owners = Owner.query();
}];
The above snippet shows how a "resource" can be created by declaring an Owner resource and then initialising it as an Owner service. The controller can then use this service to query for Owners from the RESTful endpoint. In this way, you can easily create the resources that your application require and map it easily to your business domain model. This declaration is done once only in the app.js file. You can actually take a look at this actual file in action here
 

Synchronizing states between the backend and your AngularJS application

Synchronizing states is something that needs to be managed when you are developing a client-server architecture. You will need to give some thought to how your application updates its state from the backend or refresh its view whenever some state changes.

Authentication

Having your client-side code exposed to the public makes it even more important to think through how you would like to authenticate your users and maintain session with your application. (Stateless or Stateful ?) .

Testing

AngularJS comes with the necessary tools to help you perform testing at all layers of your Javascript development from unit to functional testing. Planning how you test and perform builds incorporating those tests will determine the quality of your front end client. We use a maven plugin called frontend-maven-plugin to assist us in our build tests.

Conclusion

Migrating to AngularJS from JSP may seem daunting but it can be very rewarding in the long run as it makes for a more maintainable and testable user interface. The trend towards client side rendered views also encourages building more responsive web applications that were previously hampered by the design in server side rendering. The advent of HTML 5 and CSS3 has ushered us to a new era in View rendering technologies. The future of View layer development is on the client side and the future is now.

Thursday 12 March 2015

Authentication Mechanisms for Web Applications

Authentication is the basic requirement for most of websites. However, there are many mechanisms to implement authentication and they are not very interchangeable. Depend on business requirement, developers need to choose the most appropriate method of authentication for their application. It may not be an easy task unless one understand the differences among mechanisms well.

In this short article, I would like to recap 4 popular mechanisms for implementing authentication. There will be comparison and consideration to choose the most appropriate method as well.

Background

In a standard Java web application, there are two kinds of meta-data that normally included in request, session and authentication. The session information help to make HTTP protocol stateful while the authentication information help to identify user.

Usually, there is one header contains authentication information, another header contains session information or a single header contains both. It is also possible for a request to have only one of the two.

When HTTP is first introduced as a stateless protocol, the official authentication mechanism is basic access authentication. With this authentication method, every request must include a header that contains encrypted username and password. Apparently, basic authentication is only appropriate for a secured channel.

Still, many people would feel that basic authentication is too unsecured. Surprisingly, the basic authentication requests client to encrypt instead of hashing the password. Even worse, the encrypted password is included in every request. Therefore, if hackers manage to intercept a single request, the user password is revealed.

To strengthen the authentication mechanism, digest access authentication is introduced. With the new authentication method, client send the hashed value of combination of username, password, realm and server nonce. While realm is fixed, the nonce can be randomly generated value issued by server. Later modification of digest authentication allow client to add its nonce as well. Kindly notice that the digest authentication send the plain values of username, nonces and realm in the request. Here is one example

Authorization: Digest username="some_user", realm="Some Realm", nonce="MTQyNjE1MDE5Njc0MjoxZjdkYjIzZjI0YjZjNDExMzU2OTk3MWIyNWQzYmYwNg==", uri="/some_website/index.html", response="fcc3a3cd69c93c76e65c845263c3d4f4", qop=auth, nc=00000001, cnonce="c61b667053c03c31"

In the above request, the response value is the hashed value of username, password, realm, server nonce and client nonce combined. The server calculated its own response and authenticate user if the responses are matched.

Digest authentication make it almost impossible to figure-out the password by intercepting requests and it is also helps to prevent the replay-attack. For that to happen, the server nonce should be changed regularly. Therefore, re-sending a request after the server nonce expired will fail.

Most of readers may notice that these two methods are not that popular anymore. The HTTP based authentication mechanisms are supported by the browsers and may not provide user a beautiful login form. Worse, it provides no easy way to logout unless user close the browser or the tab. Therefore session-based login is the preferred authentication mechanism nowadays. Basic and digest authentication are still used but mostly for B2B communication where maintaining a session is not necessary.

Session-based login is applicable when HTTP is stateful. For this to happen, there is a session cookie that embedded into every request (normally, it is named JSESSIONID). The server will create a session storage for each session it created. When a request arrive, the server will try to locate user profile in user session. If the user profile is not available, a login form will be shown to user. Upon successful authentication, the server store user profile into session storage.

This method works well but it requires the stickiness load balancer if the web application is deployed in a cluster environment. Otherwise, a server will not recognize the session if it was generated by another server.

For the cloud application, it is better to build a client-side session with a cookie that contains user information. This session cookie is secured with a signature. To avoid reading database frequently, we can use a common key to sign all the sessions instead of using password.

Comparison

After knowing each mechanism, let compare them:

Features Basic Authentication Digest Authentication Server-side Session Client-Side Session
Support Session No No Yes Yes
Logout Close Browser Close Browser Kill session Kill session
Support Cluster Yes Yes Require stickiness LB Yes
Prevent Replay Attack No Yes Optional (session timeout) Optinal (include timestamp in session signature)
Client Computation Cheap Cheap Zero Zero
Server Computation Cheap Cheap Zero Cheap
Client Storage username & password username & password session cookie session cookie
Server Storage zero zero session storage zero
Handshake Not required at least 2 requests at least 2 requests at least 2 requests
Network overhead short header long header short header long header

Conclusion

You can see the above comparison and make the choice yourself. For me, Basic Authentication and Digest Authentication seem to be the better choice to build B2B authentication. However, they do not fit customer facing application well.

Choosing between Basic and Digest Authentication is merely depend on how secured is your channel. Most of the time, it is worth safeguarding our application with Digest Authentication as the network, handshake and computation overhead is not very much.

For customer facing UI, I would prefer Client-side session over Server-side session because they scale better. You always can throw in new servers to share the workload. For server-side session, the additional server can only serve new sessions, not existing sessions, unless you can clone session storages.

The problem with client-side session is the expose of user information in the session cookie. Because of that, you can only put primitive and not confidential values in the session. If you can live with that limit then client-side session is surely the better choice.

Tuesday 24 February 2015

AngularJs as the alternative choice for building web interface

Recently, we were invited to conduct a joint talk with a UX designer, Andrew from Viki on Improving communications between Design and Development for Singasug.  The main focus of the talk was to introduce AngularJS and elaborate on how it help to improve collaboration between developers and designer from both sides of view. To illustrate the topic, we work together on revamping the Pet Clinic Spring sample application.

Over one month of working remotely, mostly on spare time, we have managed to refresh the Pet Clinic application with a new interface, which built on AngularJS and Bootstrap. Based on what we have been through, we delivered the talk to share what have been done with Spring community.

In this article, we want to re-share the story with a different focus, that is using AngularJs as the alternative choice for building web application

Project kick starting

The project was initiated last year by Michael Isvy and Sergiu Bodiu, the two organizers of the Singapore Spring User Group. Michael asked our help to deliver a talk about AngularJS for the web night. He even went further by introducing us an UX designer named Andrew and nicely asked if we can collaborate on revamping the Pet Clinic application. Feeling interested on this idea, we agreed to work on the project and delivered the talk together.

We set the initial scope of the project to be one month and aimed to show as much functionality as possible rather than fully complete the application. Also, because of Andrew involvement, we take this opportunity to revamp the information architect of the website and give it with new layout.

Project Delivery

The biggest problem we have to solve is geographic location. All of us working on spare time and can afford limited face to face communication. Due to personal schedule, it is difficult to setup a common working place or time. For discussing about the project and planning, we only scheduled a weekly meeting during weekend.

To replace direct communication, we contact each other through Whatapp. We hosted the project on Github and turned on the notification feature so that each member is informed after any check-in. Even though there was a bit of concern at the beginning but each member play their part and the project progressed well.

To prepare in advance, we converted SpringMVC controllers with Jsp view to the Restful controllers. After that, Andrew showed us the wireframe flow that laid the common understanding of how the application should behave. Based on that, we built the respective controllers and integrated with html templates Andrew provided.

After half of the project time, Andrew picked up some understanding about Angular directive and be able to work directly on the project source code. That helped to boost our development speed.

When the project has been through half of the timeline, Michael came back to us and asked if it is possible to make the website functional without the Restful API. He think that would help designers to be able to develop html template without deploying it into an application server. We bought into this idea and provided this feature for the application. By turning on a flag, the AngularJS application will no longer make any http call to server. Instead, the mock http service will returned the pre-defined static Json files.

What we have achieved

After one month, we have a new website with beautiful theme that look like a commercial website rather than sample application. The new Pet Clinic is now a single page application that accessing Restful API.

We did figure out an effective way of working remotely even when we did not know each other before the project started.

We also found many benefits of building web interface with AngularJs. That is what we want to share with you today.

Understanding AngularJS

If you are new to AngularJS, think of it as a new MVC framework for front-end application. MVC is a well known pattern for server side application, but it is applicable for front-end application as well. With AngularJs, developer build a model based on the data received from server. After that, Angular will bind the model data to the html controls, displaying output on screen.

The good thing here is AngularJs provide the two-way binding instead of one-way binding. In this case, if user update the value in the html control, the model is automatically updated without developer effort.

This is the illustration of the boring one-way binding versus the two-way binding




Angular also provides scope for each model, so that the 2-way binding is only active within the boundary of a controller. To instruct the binding, AngularJs uses directives, which embedded directly to html controls. The Angularised html will look similar to this:

<div class="col-md-3" data-ng-repeat="owner in owners">
  <div class="thumbnail">
      <div class="caption">
      <h3 class="caption-heading">
           <a href="show.html" data-ng-bind="owner.firstName + ' ' +owner.lastName"></a>
      </h3>
      <p data-ng-bind="owner.city"></p>
      <p data-ng-bin="owner.address"></p>
    </div>
  </div>
</div>

For the above binding to work, you need to prepare the data for the model "owners" in your controller

var OwnerController = ['$scope','$state','Owner',function($scope,$state,Owner) {
 $scope.owners = Owner.query();
}];

If you feel curious how the code of the controller look so short, it is because we implement an Owner service with JPA liked methods

var Owner = ['$resource','context', function($resource, context) {
 return $resource(context + '/api/owners/:id');
}];

Look at the above template, if we take out all Angular directives, which start with data-ng or ng, we should have back the original html template. The only exception here is the directive data-ng-repeat, which function like a for loop that help to shorten the html code.

After Angularised the template file and create controller, the last thing one need to do is to declare them in the global app.js file

....
.state({ name: "owners",
  url: "/owners",
  templateUrl: "components/owners/owners.html",
  controller: "OwnerController",
  data: { requireLogin : true })

....
app.controller('OwnerController', OwnerController);
...
app.factory('Owner', Owner);

So far, that is the effort to Angularised one owners page template. The above example is mostly about showing data, but if we replace the tag <p/> with <input/> then users should be able to edit and view the owner at the same time.

Pros and cons of AngularJS

We have done evaluation on AngularJS before adopting it and we have evaluated it again after using. To recap our experience, we will share some benefits and issues of AngularJS.

Facilitate adoption of Restful API

Obviously, one need to introduce Restful API to work with AngularJS. Given the changes happened in this industry over last decade, Restful API is slowly but surely will be the standard practice for future applications.

In the past, a typical Spring framework developer should know JDBC, JPA, Spring, Jsp in order to develop web applications. But they are no longer enough. At first, there are some big players like Twitter, Facebook or Google that introducing the APIs for third-party web applications to integrate with their services. Later, there is a booming trend of mobile applications that no longer render UI based on html.

Because of that, there is an increasing demand for building application that serving data instead of serving html. For example, any start-up that want to play safe with will start with building Restful API before building a front-end application. That can save a lot of effort if there is a need to build for another client rather than browser in the future.

There is another contributing factor from development point of view. Splitting back-end and front-end applications make parallel development easier. It is not necessary to wait until the back-end services completed before building web interface. It is also beneficial in term of utilizing resources. In the past, we need developers that know Java and Javascript to develop java web application. Similarly, we need developers that know .NET and Javascript to develop .NET application. However, the web applications nowadays are using a lot more Javascript than in the past. That make finding developers that good at both languages harder to find. With Restful API, it is possible to recruit front-end developers that have strong understanding of Javascript and Css to build web interface while back-end developers can focus on security, scalability and performance issues.

Because we adopted Restful API from development benefit, it is important for us to showcase the ability of running Angular application without back-end service. Because AngularJS injects the modules by declaration, we have a single point of integration to configure the http service for the whole application. This look very favorable, especially for a Spring developer, because we have better control over the technology usage.

Faster development

Jsp by nature is only for viewing. Changing of data happened by other methods like form submitting or Ajax request. That why most of us think the MVC pattern only include one-way binding for displaying view. That not necessary to be the case if you remember desktop applications. For front end, MVC is not that popular. Some developers still like to manually populate values for html controls after each Ajax request. We also do but only sometimes. Some other times, we prefer automation over control. That why we love AngularJS.

As any other MVC framework, AngularJS provide a model behind the scene for each html control. But it is a Wow factor to have it reversed way, when the html control update the model. Think about it practically, sometimes our html controls are inter-related. For example, updating the value of country from the first drop down suppose to display corresponding states or provinces in the next drop down. AngularJS allows us to do this without coding.

Directive is cleaner comparing to Jsp or jQuery

I think the biggest benefit that AngularJS offer is custom directive. We adopted AngularJS because of two-way binding but custom directive is what make us committed.

Directive captured our interest at first sight. It is a clean solution for the long term problems that we face, the continuity problem and tracking problem.

Let start with a short example of a for loop in jsp file


<% for(int i = 0; i < owners.size(); i+=1) { %>

...

<% } %>

The code between opening and closing of for loop helps to render one element. Whenever we write this kind of code, we break the original html template into smaller chunks. Therefore, it is understandable that designer don't like to maintain jsp file. They are meant to be looked at by developers not designers.

To avoid inserting opening and closing of for loop at different places, we can use jsp tag. Then, we have another issue with the html code go missing in the jsp file. There is no clean and easy way for us to represent this jsp contents to designers. If there is a minor change to be made to the web interface, it normally comes as an UI patch request from designers.

Thing did not get better when developers moved to jQuery based UI. We always have concern regarding DOM manipulation and event handlers registration. In a typical jQuery web application, viewing source give us very limited information on what the user would see. Instead of being the source of truth, HTML become the material for Javascript developers to perform magic. That even shun the designers away more than Jsp.

Even for developers, thing does not go well if you are not the author of the code. It may take guess work to find out the code that being executed when user clicks a button, unless you know the convention. The Css selector is too flexible. It allows one developer to define behavior in a non-deterministic way and let the other developer go around searching for it.

Directive helps us get rid of both problems.

The directive data-ng-repeat="owner in owners" is inserted directly to the html tag. That make it is easier to keep track when the loop actually end both for us and the designers. The html inside the div remained similar to the original template, which make minor modification is possible. For tracking purpose, seeing ng-click directive on the html tags tell us immediately which method to look for. The ng-controller directive or the routing configuration gave us clue which controller that the method should be defined in. That make reaching source much easier.

Finally, the AngularJS team allow us to define our own directive. This is amazing because it allows us to create new directive to solve un-foreseen circumstance in the future and tap into existing directives library created by community.

Let get ourselves familiarized with custom directive by this example. In our Pet Clinic app, clicking some links on the banner will scroll the page down to respective part of the website.

To achieve this behavior in a jQuery app, the fastest solution is to make it as a link and put

href="javascript:scrollToTarget(destination)"

For an Angular app, it is not necessarily to be a link and the code look slightly different

ng-click="scrollToTarget(destination)"

If he decided to make a directive for this function, the code will look like

scroll-to-target="destination"

For us, the last one is the best. The directive is a familiar concept for any designer, as they should have used class, id, name or onclick. Having a new well meaning directive will not be that hard to pick up even for a designer. For us, it is fun and cool. We are no longer depend on World Wide Web consortium to release the feature that we want. If we need somethings, we can hunt for the directive, we also can modify it or create it our selves. Slowly, we can have a community-backed directives library that forever improving.

Automation versus Control

After AngularJS presentation, I think not every folk will be ready to jump the boat. The AngularJS deliver so much magic with a big trade off that you need to relinquish the control of your application over to AngularJS.

Instead of having work done yourself, you need to provide information for AngularJS to do the work. Two-way binding, injection of services all happen behind the scene. When we need to modify the way it work, it is not so convenient to do so.

For us, relinquishing control to build an configurable application is tolerable. However, not able to alter the behavior when we need is much less tolerable. This is where Spring framework shines. It does not simply give you the bean, it allow you to do whatever way you want with it like executing bean life-cycle functions, injecting bean factory or defining scope.

Contrast to this, I find it hard to go out of Angular way when I want to. For example, I would like Angular to help us populate content of a textarea and later running CkEditor Javascript to convert this textarea to a html editor. To get this done, the application need to load CkEditor Javascript only after the binding is completed. It can be done by altering AngularJS behavior or converting the textarea to a directive.

But I am not satisfied enough with both, the former solution does not look clean and the latter solution seem tedious. It look cooler if we have life-cycle support to inject additional behavior like we have with Spring framework

FAQ about AngularJS

Up to this point, I hope you already have some ideas and can make a choice for yourself whether AngularJS is the right choice. To contribute to your decision making, I will share our answers of the questions we received in the presentation.

Is it possible to protect confidential data when you build web interface with AngularJS?

For me, this is not AngularJS issue, this is how you design your Restful API. If there is something you do not want users to see, don't ever show it on your Restful API. For a Spring developer, this means avoid using entity as @ResponseBody if you are not ready to fully expose it. At least, put in some annotation like @JsonIgnore to avoid the confidential fields.

How you pass object around different pages?

There are many ways to do so. Each page have its own controller but the whole application share a single rootscope that you can place the object there. However, we should not use rootscope unless there is no other way. Our prefer solution is using url.

For example, our routing configuration specify:

       
        .state({
  name: "owners",
  url: "/owners",
  templateUrl: "components/owners/owners.html",
  controller: "OwnerController",
  data: { requireLogin : true }
 }).state({
  name: "ownerDetails",
  url: "/owners/:id",
  templateUrl: "components/owners/owner_details.html",
  controller: "OwnerDetailsController",
  data: {requireLogin : true}
 })


When user route to the owner detail page, the controller load the owner again from the id in url. It may look redundant but it allow the browser bookmarking.

Is this really possible for designers to commit to the project?

Yes, that what we have tried and it worked. One day before the presentation, our designer, Andrew redesign the landing page without our involvement. He kept the all the directives we put in and there is no impact to functionality after the design changed.

Should I adopt AngularJs 1 or wait until AngularJs 2 available?

I think you should go ahead with AngularJS 1. For us, AngularJs 2 is so different that it can be treated as another technology stack and support for AngularJS will continue for at least a year after AngularJS 2 is released (slated for 2016) We feel that due to community support, AngularJs 2 will continue to thrive like Play framework 1 after the release of Play framework 2.


Conclusion

So, we have done our part to introduce a new way of building web interface for Spring framework user. We hope you are convinced that Restful API is the way to go and AngularJs is worth trying. If you have any other idea, please help to feedback.

If you are keen to see the project or original talk, here is the reference:

https://github.com/singularity-sg/spring-petclinic

http://petclinic.hopto.org/petclinic/slides/#/

http://petclinic.hopto.org/petclinic/#/

Enjoy!