Spring Batch ItemReader Example

May 8, 2022

In this tutorial, we will see Spring Batch ItemReader Example. In Spring Batch, the ItemReader is the core interface that is used to read the data. The read() method is defined in ItemReader interface. Spring Batch handles read-write scenarios by managing an ItemReader and an ItemWriter.

The ItemReader interface contains read() method that defines the contract of the ItemReader. The read() method returns item or null if no more items are left.

public interface ItemReader {
    T read();

}

Let’s see some important ItemReader.

FlatFileItemReader – Used to read lines from the input resources(for example excel, text etc). See an example.
JdbcCursorItemReader – Used to read data from database. See an example.
JsonItemReader – Used to read json data. See an example.
StaxEventItemReader – Used to read xml data. See an example.
JdbcPagingItemReader – Used to read data from database. See an example.
JpaPagingItemReader – Used to read data from database. See an example.
StoredProcedureItemReader – Used to read data from database. See an example.

Let’s see these ItemReaders in details.

FlatFileItemReader – The FlatFileItemReader is used to reads lines from the input resources. Using FileItemReader we can read data from excel or from text.

Note – Using FileItemReader we can’t read data from database. In order to read data from database we need to use JdbcCursorItemReader or other ItemReader.

Configuring FlatFileItemReader

@EnableBatchProcessing
@Configuration
public class SpringBatchConfig {

    @Autowired
    private DataSource dataSource;

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    public FlatFileItemReader<Student> reader() {
        FlatFileItemReader<Student> itemReader = new FlatFileItemReader<>();
        itemReader.setResource(new ClassPathResource("data.csv"));
        itemReader.setLineMapper(getLineMapper());
        itemReader.setLinesToSkip(1);
        return itemReader;
    }

    private LineMapper<Student> getLineMapper() {
        DefaultLineMapper<Student> lineMapper = new DefaultLineMapper<>();
        DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();

        String[] columnsToBeInserted = new String[]{"id", "roll_number", "name"};
        int[] fields = new int[]{0, 1, 2};
        tokenizer.setNames(columnsToBeInserted);
        tokenizer.setIncludedFields(fields);
        BeanWrapperFieldSetMapper<Student> fieldSetMapper = new BeanWrapperFieldSetMapper<>();
        fieldSetMapper.setTargetType(Student.class);
        lineMapper.setLineTokenizer(tokenizer);
        lineMapper.setFieldSetMapper(fieldSetMapper);
        return lineMapper;
    }

    @Bean
    public StudentItemProcessor processor() {
        return new StudentItemProcessor();
    }

    @Bean
    public JdbcBatchItemWriter<Student> writer() {
        JdbcBatchItemWriter<Student> writer = new JdbcBatchItemWriter<>();
        writer.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>());
        writer.setSql("insert into student(id,roll_number,name) values (:id,:rollNumber,:name)");
        writer.setDataSource(dataSource);
        return writer;
    }


    @Bean
    public Job writeStudentDataIntoSqlDb() {
        JobBuilder jobBuilder = jobBuilderFactory.get("STUDENT_JOB");
        jobBuilder.incrementer(new RunIdIncrementer());
        FlowJobBuilder flowJobBuilder = jobBuilder.flow(getFirstStep()).end();
        Job job = flowJobBuilder.build();
        return job;
    }

    @Bean
    public Step getFirstStep() {
        StepBuilder stepBuilder = stepBuilderFactory.get("getFirstStep");
        SimpleStepBuilder<Student, Student> simpleStepBuilder = stepBuilder.chunk(1);
        return simpleStepBuilder.reader(reader()).processor(processor()).writer(writer()).build();
    }

}

See complete example here.

JdbcCursorItemReader – The JdbcCursorItemReader is used to read data from database.

Configuring JdbcItemReader

Annotation-based configuration


    @Bean
    public JdbcCursorItemReader<Student> reader() {
        JdbcCursorItemReader<Student> reader = new JdbcCursorItemReader<>();
        reader.setDataSource(dataSource);
        reader.setSql("select id, roll_number, name from student");
        reader.setRowMapper(new StudentResultRowMapper());
        reader.setMaxRows(10);
        reader.setFetchSize(10);
        reader.setQueryTimeout(10000);
        return reader;
    }

XML based configuration

<bean id="studentItemReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader">
<property name="dataSource" ref="datasource"/>
<property name="sql"
value="select id, roll_number, name, price from student
where name like ?"/>
<property name="rowMapper" ref="studentRowMapper"/>
//some more configuration field.

See complete example here.

JsonItemReader – The JsonItemReader is used to read json file.

Note – JsonItemReader reads data that is in below format.

[ { JSON object }, { JSON object } ]

Configuring JsonItemReader

    public JsonItemReader<Student> jsonItemReader() {
        return new JsonItemReaderBuilder<Student>()
                .jsonObjectReader(new JacksonJsonObjectReader<>(Student.class))
                .resource(new ClassPathResource("data.json"))
                .name("studentJsonItemReader")
                .build();
    }

See complete example here.

StaxEventItemReader – The StaxEventItemReader spring batch itemreader is used to read data from XML file.

Configuring StaxEventItemReader.

    public StaxEventItemReader xmlItemReader() {
        return new StaxEventItemReaderBuilder<Student>()
                .name("itemReader")
                .resource(new ClassPathResource("data.xml"))
                .addFragmentRootElements("student")
                .unmarshaller(studentMarshaller())
                .build();

    }

    public XStreamMarshaller studentMarshaller() {
        Map<String, Class> aliases = new HashMap<>();
        aliases.put("student", Student.class);
        aliases.put("id", Long.class);
        aliases.put("rollNumber", String.class);
        aliases.put("name", String.class);
        XStreamMarshaller marshaller = new XStreamMarshaller();
        marshaller.setAliases(aliases);
        return marshaller;
    }

See complete example here.

JdbcPagingItemReader – The JdbcPagingItemReader itemreader is used to read data from database.

Configuring JdbcPagingItemReader.

 @Bean
    public JdbcPagingItemReader<Student> jdbcPagingItemReader() {
        JdbcPagingItemReader<Student> pagingItemReader = new JdbcPagingItemReader<>();

        pagingItemReader.setDataSource(dataSource);
        pagingItemReader.setFetchSize(20);
        pagingItemReader.setRowMapper(new StudentResultRowMapper());
        pagingItemReader.setPageSize(1);

        MySqlPagingQueryProvider mySqlPagingQueryProvider = new MySqlPagingQueryProvider();
        mySqlPagingQueryProvider.setSelectClause("id, roll_number, name");
        mySqlPagingQueryProvider.setFromClause("from student");

        Map<String, Order> orderByName = new HashMap<>();
        orderByName.put("name", Order.ASCENDING);

        mySqlPagingQueryProvider.setSortKeys(orderByName);
        pagingItemReader.setQueryProvider(mySqlPagingQueryProvider);

        return pagingItemReader;
    }

See a complete JdbcPagingItem example using MySql database.

JpaPagingItemReader – The JpaPagingItemReader is used read data from database. We can write JPQL to reads records from Database.

Configuring JpaPagingItemReader.

  @Bean
    public JpaPagingItemReader<Student> getJpaPagingItemReader()  {
        String sql = "select * from student where id >= :limit";
        JpaNativeQueryProvider<Student> queryProvider = new JpaNativeQueryProvider<Student>();
        JpaPagingItemReader<Student> reader = new JpaPagingItemReader<>();
        queryProvider.setSqlQuery(sql);
        reader.setQueryProvider(queryProvider);
        queryProvider.setEntityClass(Student.class);
        reader.setParameterValues(Collections.singletonMap("limit", 10));
        reader.setEntityManagerFactory(entityManagerFactory);
        reader.setPageSize(3);
        reader.setSaveState(true);
        return reader;
    }

See JpaPagingItemReader example using MySql.

StoredProcedureItemReader – Using StoredProcedureItemReader item reader we can write stored procedure and read data from database.

Configuring StoredProcedureItemReader

Define RowMapper for StoredProcedureItemReader

public class StudentResultRowMapper implements RowMapper<Student> {
    @Override
    public Student mapRow(ResultSet rs, int i) throws SQLException {
        Student student = new Student();
        student.setId(rs.getLong("id"));
        student.setRollNumber(rs.getString("roll_number"));
        student.setName(rs.getString("name"));
        return student;
    }
}

Configure StoredProcedureItemReader

    @Bean
    public StoredProcedureItemReader reader() {
        StoredProcedureItemReader reader = new StoredProcedureItemReader();
        reader.setDataSource(dataSource);
        reader.setProcedureName("student_procedure");
        reader.setRowMapper(new StudentResultRowMapper());
        reader.setRefCursorPosition(1);
        return reader;
    }

What would return by Read method of ItemReader?

The return type of read() method is T. It returns item if data is available in resource else it returns null.

What would return by Read method of ItemReader interface if all input data has been exhausted from the source?

null

Is ItemReader mandatory component in Spring Batch?

Yes. IteamReader and IteamWriter is mandatory component in Spring Batch.

That’s all about Spring Batch ItemReader Example.

Download example from github.