Spring Boot 2.0 was recently released, and I decided I would investigate the amount of work required to migrate. This article is the write up of that experience.
The project that I worked on is a medium-sized multi-maven project with 50,000 lines of Java (as reported by sloccount), including several library modules, resulting in three separate war files.
Before you begin the migration, you should read the official migration guide.
Spring Data JPA repository methods
Error:
The first problem I found was the following compile errors. The errors themselves were slightly confusing due to the mention of QueryByExampleExecutor, an interface I didn’t know I was using.
error: method findOne in interface QueryByExampleExecutor<T#2> cannot be applied to given types;
return getRepository().findOne(id);
error: method exists in interface QueryByExampleExecutor<T#2> cannot be applied to given types;
return getRepository().exists(id);
error: method save in interface CrudRepository<T#2,ID> cannot be applied to given types;
Iterable savedObjects = getRepository().save(objects);
error: incompatible types: PK cannot be converted to T
getRepository().delete(id);
Solution:
Spring Data Commons 2.0.0 changed the name of methods in CrudRepository to have a more consistent naming scheme:
- Methods referring to an identifier now all end on …ById(…).
- Methods taking or returning a collection are named …All(…)
That results in the following renames:
- findOne(…) -> findById(…)
- save(Iterable) -> saveAll(Iterable)
- findAll(Iterable) -> findAllById(…)
- delete(ID) -> deleteById(ID)
- delete(Iterable) -> deleteAll(Iterable)
- exists() -> existsById(…)
This change required that I modified everywhere that called these methods. Fortunately, because I used the Manager/Repository pattern, the access was limited to the Manager layer (and tests that called the repositories directly).
Future Work:
- I should consider using a consistent naming scheme for my manager method names
- I should audit my tests to make sure they use the manager layer rather than repository layer when they aren’t testing the repository layer.
More Information:
- https://github.com/spring-projects/spring-data-commons/commit/727ab8384cac21e574526f325e19d6af92b8c8df#diff-2113692347b2c25fc21d27f8fc93f876
- https://jira.spring.io/browse/DATACMNS-944
Flyway property names:
Error:
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Invocation of init method failed; nested exception is org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateSqlException:
Migration V1.1.0_4__DDC-91_add_useContext_to_rulesets.sql failed
----------------------------------------------------------------
SQL State : 42501
Error Code : -5501
Message : user lacks privilege or object not found: PUBLIC.SAMPLE_PAGE_RULESET
Location : db/migration/V1.1.0_4__DDC-91_add_useContext_to_rulesets.sql
Line : 1
Statement : ALTER TABLE Sample_Page_Ruleset add use_context VARCHAR(150)
Solution:
This error was easy to resolve, and was mentioned in the Migration Notes. https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-2.0-Migration-Guide#flyway
My tests use HSQLDB as the underlying database engine to remove external dependencies and to allow an in-memory database for performance. Because of this, we use Hibernate’s auto-DDL functionality to create the database. But it also means that we don’t want or need Flyway to run database migration scripts. It fails if it does run, because the schema is already the latest version. To solve this, we were using flyway.enabled=false in the application-test.properties to selectively disable flyway during the automated tests.
The flyway configuration namespace had moved from flyway.* to spring.flyway.*, meaning we just had to change our application-test.properties to use spring.flyway.enabled=false. With this change, flyway no longer ran any migrations during tests.
Future Work:
None
SQL script missing a default identifier
Error:
Failed to execute SQL script statement #38 of class path resource [sql/editions.sql]: INSERT INTO Bookmark (edition_id, page_no, depth, bookmarkText) VALUES (10, 1, 1, 'Front Cover'); nested exception is java.sql.SQLIntegrityConstraintViolationException: integrity constraint violation: NOT NULL check constraint; SYS_CT_10158 table: BOOKMARK column: ID
Solution:
This error was fairly easy to resolve. We use the @Sql annotation to load test data into the database. One of them had previously relied on the table to have an auto-assigned primary key column, but a change in Hibernate 5 seems to have changed this behaviour.
The schema for the table generated by Hibernate 5.2.14.Final was:
[sourcecode lang=”sql”]create table Bookmark (
id bigint not null,
last_update timestamp,
bookmarkText varchar(5000) not null,
depth integer not null,
page_no integer,
edition_id bigint,
primary key (id)
)[/sourcecode]while under Spring Boot 1.5.10 and with Hibernate 5.0.12.Final was:
[sourcecode lang=”sql”]create table Bookmark (
id bigint generated by default as identity (start with 1),
last_update timestamp,
bookmarkText varchar(5000) not null,
depth integer not null,
page_no integer,
edition_id bigint,
primary key (id)
)[/sourcecode]The fix was to add id values into the SQL script.
[sourcecode lang=”sql”]INSERT INTO Bookmark (id, edition_id, page_no, depth, bookmarkText) VALUES (1, 10, 1, 1, ‘Front Cover’);[/sourcecode]This change is possibly related to the issue below about how Hibernate generates id values, but this issue came up first, so was solved before the generator issue appeared. It’s also possibly a good idea to hardcode the ids for test data so that we aren’t surprised by ids changing and causing errors in our tests.
Future Work:
None
Unflushed Hibernate tests
Error:
A number of unit tests were failing with:
Expected exception: org.springframework.dao.DataIntegrityViolationException
Solution:
In our case, we were intentionally expecting an integrity violation, but weren’t receiving one. We were inserting two objects with the same id in an unique column. Unfortunately the two insert queries were not being executed by the time the test ended.
When you call EntityManager.persist() or EntityManager.merge() to save your objects to the database, it doesn’t automatically run the INSERT query straight away. It can happen some time later. This unpredictable behaviour is not ideal for unit tests, so it’s best to call EntityManager.flush() before you want to verify the state of your database. Because we are using Spring Data JPA, it required changing our repository.save(object) method to repository.saveAndFlush(object). There is a separate repository.flush() method, but it makes sense to merge the two calls into one. We already do this in our Manager layer because the application is not write heavy and we are happy not to have the possible performance optimisation that Hibernate might use.
Hibernate defaulting to generating IDs using sequences
Error:
- A different object with the same identifier value was already associated with the session
- could not execute statement; SQL [n/a]; constraint [SYS_PK_10260]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement: integrity constraint violation: unique constraint or index violation; SYS_PK_10260 table: IMPRINT
Solution:
This took a while to figure out. We had a unit test that would load some test data from an SQL script and then insert some new entities into the database. This worked under Spring Boot 1.5.10/Hibernate 5.0.12.Final, but suddenly wasn’t working now. The clue to the issue was found in the SQL being run:
2018-03-11 15:03:00.523 INFO 25658 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executing SQL script from class path resource [sql/base-data.sql]
2018-03-11 15:03:00.709 INFO 25658 --- [ main] o.s.jdbc.datasource.init.ScriptUtils : Executed SQL script from class path resource [sql/base-data.sql] in 186 ms.
...
2018-03-11 15:03:00.859 DEBUG 25658 --- [ main] org.hibernate.SQL : call next value for hibernate_sequence
2018-03-11 15:03:00.886 INFO 25658 --- [ main] c.s.r.d.m.s.c.i.AvailabilityUpdateAspect : Updated imprint: id: 1 name: 'Imprint 0' - added event to availability update queue
2018-03-11 15:03:00.889 DEBUG 25658 --- [ main] org.hibernate.SQL : call next value for hibernate_sequence
2018-03-11 15:03:00.891 INFO 25658 --- [ main] c.s.r.d.m.s.c.i.AvailabilityUpdateAspect : Updated imprint: id: 2 name: 'Imprint 1' - added event to availability update queue
2018-03-11 15:03:00.892 DEBUG 25658 --- [ main] org.hibernate.SQL : call next value for hibernate_sequence
2018-03-11 15:03:00.895 INFO 25658 --- [ main] c.s.r.d.m.s.c.i.AvailabilityUpdateAspect : Updated imprint: id: 3 name: 'Imprint 2' - added event to availability update queue
2018-03-11 15:03:00.898 DEBUG 25658 --- [ main] org.hibernate.SQL : call next value for hibernate_sequence
2018-03-11 15:03:00.900 INFO 25658 --- [ main] c.s.r.d.m.s.c.i.AvailabilityUpdateAspect : Updated imprint: id: 4 name: 'Imprint 3' - added event to availability update queue
2018-03-11 15:03:00.900 DEBUG 25658 --- [ main] org.hibernate.SQL : call next value for hibernate_sequence
2018-03-11 15:03:00.901 INFO 25658 --- [ main] c.s.r.d.m.s.c.i.AvailabilityUpdateAspect : Updated imprint: id: 5 name: 'Imprint 4' - added event to availability update queue
2018-03-11 15:03:00.945 DEBUG 25658 --- [ main] org.hibernate.SQL : insert into Imprint (last_update, external_id, pending, available_days_before_published, comment, description, enable_available_before_published, name, publisher_id, use_cbmc, id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
As you can see, the IDs for the new objects was 1 to 5, but we already a row with that ID in the database from the test data in the SQL script. The thing that stood out was:
org.hibernate.SQL : call next value for hibernate_sequence
Hibernate was using a single named sequence for generating ID values. That was different behaviour than previously, and would cause problems with our existing data in the database. We’d have to make sure that the sequence had a value higher than any table in the database.
Our entity primary keys were annotated with:
[sourcecode lang=”java5″]@GeneratedValue(strategy = GenerationType.AUTO)[/sourcecode]which let Hibernate choose. Previously, it would fall back to asking if the dialect supported identities and using them and falling back to sequences (including a fall back to simulated support using tables).
The solution was to revert to the previous behaviour. There was two options: setting hibernate.id.new_generator_mappings to false, as was the default in Spring Boot 1.5 (but not the default in Hibernate 5), or change the mapping to use the identity generator type explicitly. I’d rather not use settings that differ from Hibernate defaults, so I decided to pick the latter option.
This required changing the id mapping from:
[sourcecode lang=”java5″]@Id
@GeneratedValue(strategy = GenerationType.AUTO)
protected Long id;[/sourcecode]to
[sourcecode lang=”java5″]@Id
@GeneratedValue(strategy = GenerationType.AUTO, generator = “native”)
@GenericGenerator(name = “native”, strategy = “native”)
protected Long id;[/sourcecode]
More Information:
- https://docs.jboss.org/hibernate/orm/5.2/userguide/html_single/Hibernate_User_Guide.html#identifiers-generators
- https://github.com/spring-projects/spring-boot/issues/7612
- https://github.com/spring-projects/spring-boot/commit/bc1ee76b557e51e16bb31facecb1c049ed63322f
Future Work:
Migrate to using sequences, as there are performance improvements from using sequences, including the ability to use batch inserts. This is something we’ve identified as a slow down in a number of requests. The main thing we’d need to investigate is how to make sure the sequences are sufficiently high enough for the data in the database and to make sure that this includes with test data.
Left overs from a Log4j -> SLF4J migration
Error:
error: package org.apache.log4j does not exist
import org.apache.log4j.Logger;
error: cannot find symbol
private static final Logger logger = Logger.getLogger(ImageExtractor.class);
Solution:
This was confusing because I thought we’d completed the migration to SLF4J some years ago, but clearly some log4j usage still existed in the code base, and the dependencies had been hiding those instances. SLF4J’s Log4J bridge didn’t help expose those errors. It seems that Spring Boot 2.0 has changed the dependencies to not include Log4J, hence the compile errors.
The fix was to migrate those instances to SLF4J.
Dependency issues
Error:
Could not resolve dependencies for project com.example:web-service:war:2.0.0-SNAPSHOT: Failure to find com.example:old-library:jar:2.0.0-SNAPSHOT
Solution:
This wasn’t really related to Spring, but the project version bump exposed an old sub-module dependency that was still included. That just required removing the offending dependency.
However I took this opportunity to audit our dependency versioning, and specifically, making sure we didn’t have any versions that differed from the Spring Boot dependency management list. This helps us not suffer any library incompatibilities between our versions and the blessed versions from Spring Boot.
The first step was to remove all of the version numbers from the submodules’ pom.xml and including them in parent pom’s section. Once all the version information was the top level pom, I removed all the unused libraries listed in the dependencyManagement section, leaving just the used libraries. Finally, I went through and removed any entry that was also included in the Spring Boot parent pom. Fortunately, Spring Boot lists all their dependencies in the reference manual.
More Information:
- https://docs.spring.io/spring-boot/docs/current/reference/html/appendix-dependency-versions.html
SprintBootServletInitializer package move
Error:
error: package org.springframework.boot.web.support does not exist
import org.springframework.boot.web.support.SpringBootServletInitializer;
error: cannot find symbol
public class WidgetApplication extends SpringBootServletInitializer {
Solution:
The class changed to the org.springframework.boot.web.servlet.support package, so changing the import line resolved the problem.
SecurityProperties.ACCESS_OVERRIDE_ORDER disappeared
Error:
error: cannot find symbol
@Order(SecurityProperties.ACCESS_OVERRIDE_ORDER)
Solution:
SecurityProperties no longer defines the ACCESS_OVERRIDE_ORDER constant for the @Order annotation. However, Spring Boot no longer defines any security details if the application does, so we do not need the @Order annotation on the security @Configuration class and can be removed.
Configuring PasswordEncoder
Error:
java.lang.IllegalArgumentException: There is no PasswordEncoder mapped for the id "null"
Solution:
Spring Security 5.0 introduced a new scheme for password encryption. It now uses a DelegatingPasswordEncoder. Rather than just storing the hashed password, it includes the password hash method at the start of the string. Unfortunately, if that hash information is missing, it has no default method and throws an exception. The solution is to define your own PasswordEncoder bean
[sourcecode lang=”java5″]@Bean
public PasswordEncoder passwordEncoder() {
PasswordEncoder passwordEncoder = PasswordEncoderFactories.createDelegatingPasswordEncoder();
((DelegatingPasswordEncoder)passwordEncoder).setDefaultPasswordEncoderForMatches(NoOpPasswordEncoder.getInstance());
return passwordEncoder;
}[/sourcecode]
More Information:
- http://info.michael-simons.eu/2018/01/13/spring-security-5-new-password-storage-format/
Future Work:
You may have noticed that the default is NoOpPasswordEncoder. Yes, embarrassingly, this system uses plain text passwords currently. The new scheme makes it very easy to implement migrations to new schemes. One of the more urgent jobs will be to use the new functionality to make sure we can transparently switch to non-plain text passwords.
Selenium Annotation problems
Error:
Caused by: org.springframework.beans.factory.BeanDefinitionStoreException: Failed to read candidate component class: file [/home/david/Work/example/admin-service/target/test-classes/com/example/admin/selenium/AssignmentIT.class]; nested exception is java.lang.ArrayStoreException
at org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider.scanCandidateComponents(ClassPathScanningCandidateComponentProvider.java:454) ~[spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE]
Solution:
This error took some time to resolve, because the error message does not give a clear idea of what the problem is. Some searching revealed that the error was down to a missing annotation definition at runtime (think java.lang.NoClassDefFoundError for annotations). The only different annotations used in this class is a couple of annotations for running Selenium tests. As we currently don’t run the selenium tests, the simplest thing to do is to comment out the annotations. This is a cop-out, but it allows us to continue with the migration. A proper fix would involve debugging why the annotation class file is missing at runtime. A Maven dependency issue would be the first place to investigate.
Cache Control tests
Error:
Failed unit tests with:
Response header 'Cache-Control' expected:<no-cache, no-store, max-age=0, must-revalidate> but was:<no-cache, must-revalidate>
Solution:
Spring 5 changed the way that the CacheControl utility class creates the header value. In particular, requesting “no-cache” no longer includes “no-store” as well. This change results in better standards-compliant cache control headers.
In this particular instance, I took this opportunity to check to see if our cache control header was correct, but the general fix was to update the expected string in the tests to match the actual value.
Response status message changed
Error:
MockMvc tests failing with:
Response status reason
Solution:
We had some MockMvc tests that were checking for 403 error responses, using:
[sourcecode lang=”java5″].andExpect(status().isForbidden())
.andExpect(status().reason(containsString(“Access is denied”)));[/sourcecode]Unfortunately the reason string changed between Spring Security 4 and Spring Security 5 from “Access is denied” to “Forbidden”. The fix was to modify the test to search for the new string instead. It’s possible that checking the reason string is not needed if we’re checking for the status code, but we’ll keep it for now.
Content Negotiation configuration problems
Error:
Test failing with:
ImageControllerTest): Content type expected:<image/jpeg> but was:<image/png>
Solution:
We have a controller that returns an image, but uses BufferedImageHttpMessageConverter to automatically return the right format based on the clients’ request. The way Spring uses to work out what format to use is down to a couple of methods. The default is to use the Accepts header, but you can also use the url file extension (which is discouraged due to security concerns) and using a parameter (which defaults to “format”). We were using the parameter method and configured it using the WebMvcConfigurerAdapter:
[sourcecode lang=”java5″]@Override
public void configureContentNegotiation(ContentNegotiationConfigurer configurer) {
configurer.favorPathExtension(false)
.favorParameter(true)
.mediaType(“jpeg”, MediaType.IMAGE_JPEG)
.mediaType(“png”, MediaType.IMAGE_PNG);
}[/sourcecode]All the documentation suggested that this should continue to work in Spring Boot 2.0. After some debugging, I discovered that the issue was because WebMvcAutoConfiguration was running after my configuration, and a change to enable content negotiation via properties was overwriting my settings. No value for the @Order annotation changed the order that the two configuration classes ran. I ended up filling a bug with Spring Boot and setting the options via the properties file:
spring.mvc.contentnegotiation.favor-parameter=true
spring.mvc.contentnegotiation.media-types.jpeg=image/jpeg
spring.mvc.contentnegotiation.media-types.png=image/png
This should be fixed in Spring Boot 2.0.1
Note: Thanks to interfaces gaining default methods in Java 8, you configuration class should implement WebMvcConfigurer rather than extend WebMvcConfigurerAdapter, and the latter has been deprecated.
Further information:
- https://github.com/spring-projects/spring-boot/issues/11105
- https://github.com/spring-projects/spring-boot/issues/12389
Future Work:
Once Spring Boot 2.0.1 is released, I can remove the properties, as the java code will work correctly.
Mockito API changes
Error:
error: no interface expected here
private static class ValidPageArgumentMatcher extends ArgumentMatcher {
Solution:
One of our tests creates a custom Mockito argument matcher, but the API changed between 1.10.19 and 2.15.0 (the versions used by Spring Boot 1.5.10 and 2.0 respectively). In particular, ArgumentMatcher went from being an abstract class to an interface. That meant that I had to change
[sourcecode lang=”java5″]private static class ValidPageArgumentMatcher extends ArgumentMatcher {[/sourcecode]to
[sourcecode lang=”java5″]private static class ValidPageArgumentMatcher implements ArgumentMatcher {[/sourcecode]They also took the opportunity to use generics in the match function arguments, so I had to change
[sourcecode lang=”java5″]@Override
public boolean matches(Object o) {
int page = (Integer) o;
…
}[/sourcecode]to
[sourcecode lang=”java5″]@Override
public boolean matches(Integer page) {
…
}[/sourcecode]
Multiple EhCache CacheManagers
Error:
Caused by: net.sf.ehcache.CacheException: Another unnamed CacheManager already exists in the same VM. Please provide unique names for each CacheManager in the config or do one of following:
1. Use one of the CacheManager.create() static factory methods to reuse same CacheManager with same name or create one if necessary
2. Shutdown the earlier cacheManager before creating new one with same name.
Solution:
I’d seen this before, but the same code worked under Spring Boot 1.5. The issue was that Hibernate was trying to set up an unnamed CacheManager when one already existed. The solution was to add the following setting:
spring.jpa.properties.hibernate.cache.region.factory_class: org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory
SingletonEhCacheRegionFactory reuses any existing CacheManager rather than creating its own.
Mockito Strict Unnecessary Stubbing Checking
Error:
org.mockito.exceptions.misusing.UnnecessaryStubbingException:
Solution:
Mockito has become stricter about stubbed method calls that don’t get called during the test. This is useful if your code suddenly doesn’t call a method that it should. In my case, code was refactored and the test not updated. I just had to remove the unneeded when() line.
Conclusion
Upgrading from Spring Boot 1.5 to 2.0 involves more work that that listed in the Migration Guide, which is not surprising, considering it’s not just a migration of Spring Boot, but also a migration of Spring, Hibernate and many more libraries.
I’m sure I have more issues to discover once I get the application onto the QA environment, but getting to a stage where all our automated tests passed was probably at most a day and a half.
Creating Windows Service for Spring Boot