Android Augmented Reality – Android ARCore Example

In this tutorial, we’ll be discussing Augmented Reality and implement a hello world sample in our android application using Sceneform.

Augmented Reality

Augmented Reality is gaining more popularity day by day. It’s used to show you virtual 3d models/graphics in a real world through your mobile’s camera. It’s useful in demonstrating simulated stuff in the real world such as paints/decoration you’d like to see on your wall.

Augmented Reality lies somewhere between Virtual Reality and Real environment.

Android ARCore

Google has come up with ARCore library to enhance AR development.

ARCore does motion tracking in the real world through the camera to create paths on planes/surfaces over which we can place our 3D Models and graphics.

SceneForm is a 3D framework that’s come up recently and is a better alternative to OpenGL. OpenGL looks intimidating to code.

SceneForm allows us to quickly render 3d objects without the need to learn graphics or OpenGL.

We can download the Google Sceneform Tools plugin in our Android Studio to view and render the 3d models.

You can go to https://poly.google.com/ and download a sample model. Don’t forget to credit the creator!

Typically, the OBJ and GLTX formats are used for rendering augmented images.

Now let’s build our first AR application where we’ll use the above 3D model as our AR image.

Augmented Reality Android Example Project Structure

In the above project, we’ve created a sampledata directory, where we add the obj and mtl file downloaded earlier.

Add the following dependency in your project’s build.gradle:

classpath 'com.google.ar.sceneform:plugin:1.0.1'

Add the following in your app’s build.gradle :

implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.0.0'

You need to set the ar plugin in your app’s Gradle file as well.
Add the following below the dependencies:

apply plugin: 'com.google.ar.sceneform.plugin'

We have used the following image, you can download it from this link and put it in the sampledata directory:

Now right click on the obj file and choose import sceneform assets:

This would automatically create the Renderable AR sceneform assets. It would automatically add the following lines at the end of the app’s build.gradle:

sceneform.asset('sampledata/Coffee Cup_final.obj',
        'default',
        'sampledata/Coffee Cup_final.sfa',
        'src/main/assets/Coffee Cup_final')

The build.gradle of the app looks like this finally:

Android ARCore Sceneform requires Java 8 or higher version.

The sja and sjb are the Sceneform Asset Description and Sceneform Binary files.

The sjb file is visible in the 3D viewer. It is shipped with the APK.

The sja file is used to set properties for the sjb file.

Android Augmented Reality Example Code

To configure ARCore in your application add the following permission and metadata to your AndroidManifest.xml file.

<uses-permission android:name="android.permission.CAMERA" />
    <uses-feature
        android:name="android.hardware.camera.ar"
        android:required="true" />

metadata goes inside the application tag.

<meta-data
            android:name="com.google.ar.core"
            android:value="required" />

The code for the activity_main.xml class is given below:

<?xml version="1.0" encoding="utf-8"?>
<android.support.design.widget.CoordinatorLayout xmlns:android="https://schemas.android.com/apk/res/android"
    xmlns:app="https://schemas.android.com/apk/res-auto"
    xmlns:tools="https://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <android.support.design.widget.AppBarLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:theme="@style/AppTheme.AppBarOverlay">

        <android.support.v7.widget.Toolbar
            android:id="@+id/toolbar"
            android:layout_width="match_parent"
            android:layout_height="?android:attr/actionBarSize"
            android:background="?android:attr/colorPrimary"
            app:popupTheme="@style/AppTheme.PopupOverlay" />

    </android.support.design.widget.AppBarLayout>

    <include layout="@layout/content_main" />

    <android.support.design.widget.FloatingActionButton
        android:id="@+id/fab"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="bottom|end"
        android:layout_margin="@dimen/fab_margin"
        app:srcCompat="@android:drawable/ic_input_add" />

</android.support.design.widget.CoordinatorLayout>

The code for the content_main.xml is given below:

<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="https://schemas.android.com/apk/res/android"
    xmlns:app="https://schemas.android.com/apk/res-auto"
    xmlns:tools="https://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:layout_behavior="@string/appbar_scrolling_view_behavior"
    tools:context=".MainActivity"
    tools:showIn="@layout/activity_main">

    <fragment
        android:id="@+id/sceneForm"
        android:name="com.google.ar.sceneform.ux.ArFragment"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

</android.support.constraint.ConstraintLayout>

We’ve set the fragment as ArFragment. The ArFragment checks if the phone is compatible with ARCore.

Also, it checks if the camera permission is granted. If it isn’t, it’ll ask for it automatically.

Also, it’ll ask you to download the ARCore by Google Application.

The code for MainActivity.java class is given below:

package com.journaldev.androidarsceneform;

import android.graphics.Point;
import android.net.Uri;
import android.os.Bundle;
import android.support.annotation.Nullable;
import android.support.design.widget.FloatingActionButton;
import android.support.v7.app.AlertDialog;
import android.support.v7.app.AppCompatActivity;
import android.support.v7.widget.Toolbar;
import android.view.View;
import android.view.Menu;
import android.view.MenuItem;

import com.google.ar.core.Anchor;
import com.google.ar.core.Frame;
import com.google.ar.core.HitResult;
import com.google.ar.core.Plane;
import com.google.ar.core.Trackable;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.ux.ArFragment;
import com.google.ar.sceneform.ux.TransformableNode;

import java.util.List;
import java.util.function.Consumer;
import java.util.function.Function;

public class MainActivity extends AppCompatActivity {

    ArFragment arFragment;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        Toolbar toolbar = (Toolbar) findViewById(R.id.toolbar);
        setSupportActionBar(toolbar);

        FloatingActionButton fab = (FloatingActionButton) findViewById(R.id.fab);
        fab.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                addObject(Uri.parse("Coffee Cup_Final.sfb"));
            }
        });

        arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.sceneForm);
    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        // Inflate the menu; this adds items to the action bar if it is present.
        getMenuInflater().inflate(R.menu.menu_main, menu);
        return true;
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        // Handle action bar item clicks here. The action bar will
        // automatically handle clicks on the Home/Up button, so long
        // as you specify a parent activity in AndroidManifest.xml.
        int id = item.getItemId();

        //noinspection SimplifiableIfStatement
        if (id == R.id.action_settings) {
            return true;
        }

        return super.onOptionsItemSelected(item);
    }

    private void addObject(Uri parse) {
        Frame frame = arFragment.getArSceneView().getArFrame();
        Point point = getScreenCenter();
        if (frame != null) {
            List<HitResult> hits = frame.hitTest((float) point.x, (float) point.y);

            for (int i = 0; i < hits.size(); i++) {
                Trackable trackable = hits.get(i).getTrackable();
                if (trackable instanceof Plane && ((Plane) trackable).isPoseInPolygon(hits.get(i).getHitPose())) {
                    placeObject(arFragment, hits.get(i).createAnchor(), parse);
                }
            }
        }
    }

    private final void placeObject(final ArFragment fragment, final Anchor createAnchor, Uri model) {
        ModelRenderable.builder().setSource(fragment.getContext(), model).build().thenAccept((new Consumer() {
            public void accept(Object var1) {
                this.accept((ModelRenderable) var1);
            }

            public final void accept(ModelRenderable it) {
                if (it != null)
                    MainActivity.this.addNode(arFragment, createAnchor, it);
            }
        })).exceptionally((new Function() {
            public Object apply(Object var1) {
                return this.apply((Throwable) var1);
            }

            @Nullable
            public final Void apply(Throwable it) {
                AlertDialog.Builder builder = new AlertDialog.Builder(MainActivity.this);
                builder.setMessage(it.getMessage()).setTitle("error!");
                AlertDialog dialog = builder.create();
                dialog.show();
                return null;
            }
        }));
    }

    private void addNode(ArFragment fragment, Anchor createAnchor, ModelRenderable renderable) {

        AnchorNode anchorNode = new AnchorNode(createAnchor);
        TransformableNode transformableNode = new TransformableNode(fragment.getTransformationSystem());
        transformableNode.setRenderable(renderable);
        transformableNode.setParent(anchorNode);
        fragment.getArSceneView().getScene().addChild(anchorNode);
        transformableNode.select();
    }

    private Point getScreenCenter() {
        View vw = findViewById(android.R.id.content);
        return new Point(vw.getWidth() / 2, vw.getHeight() / 2);
    }
}

The anchor is the position where the model or the Node is placed in the screen. It’s the center of the screen here.

In the addObject() the ARFragment gets the hit points on the plane using motion tracking.

placeObject() asynchronously sets the node to the center of the screen.

Following are the outputs when the application is launched:

In the final output, the cup is positioned on the plane which consists of white dots.

The above screenshot is from Samsung S8.

Now try changing the model scale in SFA file!

This brings an end to this tutorial. You can download the final AndroidArSceneform Project from the link below.

The size of the project is big, thanks to the OBJ file.