Android Augmented Reality – Android ARCore Example

Filed Under: Android

In this tutorial, we’ll be discussing Augmented Reality and implement a hello world sample in our android application using Sceneform.

Augmented Reality

Augmented Reality is gaining more popularity day by day. It’s used to show you virtual 3d models/graphics in a real world through your mobile’s camera. It’s useful in demonstrating simulated stuff in the real world such as paints/decoration you’d like to see on your wall.

Augmented Reality lies somewhere between Virtual Reality and Real environment.

Android ARCore

Google has come up with ARCore library to enhance AR development.

ARCore does motion tracking in the real world through the camera to create paths on planes/surfaces over which we can place our 3D Models and graphics.

SceneForm is a 3D framework that’s come up recently and is a better alternative to OpenGL. OpenGL looks intimidating to code.

SceneForm allows us to quickly render 3d objects without the need to learn graphics or OpenGL.

We can download the Google Sceneform Tools plugin in our Android Studio to view and render the 3d models.

You can go to https://poly.google.com/ and download a sample model. Don’t forget to credit the creator!

Typically, the OBJ and GLTX formats are used for rendering augmented images.

Now let’s build our first AR application where we’ll use the above 3D model as our AR image.

Augmented Reality Android Example Project Structure

Augmented Reality Android App Example

In the above project, we’ve created a sampledata directory, where we add the obj and mtl file downloaded earlier.

Add the following dependency in your project’s build.gradle:


classpath 'com.google.ar.sceneform:plugin:1.0.1'

Add the following in your app’s build.gradle :


implementation 'com.google.ar.sceneform.ux:sceneform-ux:1.0.0'

You need to set the ar plugin in your app’s Gradle file as well.
Add the following below the dependencies:


apply plugin: 'com.google.ar.sceneform.plugin'

We have used the following image, you can download it from this link and put it in the sampledata directory:
android poly google image

Now right click on the obj file and choose import sceneform assets:
android sceneform import path

This would automatically create the Renderable AR sceneform assets. It would automatically add the following lines at the end of the app’s build.gradle:


sceneform.asset('sampledata/Coffee Cup_final.obj',
        'default',
        'sampledata/Coffee Cup_final.sfa',
        'src/main/assets/Coffee Cup_final')

The build.gradle of the app looks like this finally:
android ARCore app build gradle

Android ARCore Sceneform requires Java 8 or higher version.

The sja and sjb are the Sceneform Asset Description and Sceneform Binary files.

The sjb file is visible in the 3D viewer. It is shipped with the APK.

The sja file is used to set properties for the sjb file.

You can change the scale of the model in the sja file.

Android Augmented Reality Example Code

To configure ARCore in your application add the following permission and metadata to your AndroidManifest.xml file.


<uses-permission android:name="android.permission.CAMERA" />
    <uses-feature
        android:name="android.hardware.camera.ar"
        android:required="true" />

metadata goes inside the application tag.


<meta-data
            android:name="com.google.ar.core"
            android:value="required" />

The code for the activity_main.xml class is given below:


<?xml version="1.0" encoding="utf-8"?>
<android.support.design.widget.CoordinatorLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    tools:context=".MainActivity">

    <android.support.design.widget.AppBarLayout
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:theme="@style/AppTheme.AppBarOverlay">

        <android.support.v7.widget.Toolbar
            android:id="@+id/toolbar"
            android:layout_width="match_parent"
            android:layout_height="?android:attr/actionBarSize"
            android:background="?android:attr/colorPrimary"
            app:popupTheme="@style/AppTheme.PopupOverlay" />

    </android.support.design.widget.AppBarLayout>

    <include layout="@layout/content_main" />

    <android.support.design.widget.FloatingActionButton
        android:id="@+id/fab"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_gravity="bottom|end"
        android:layout_margin="@dimen/fab_margin"
        app:srcCompat="@android:drawable/ic_input_add" />

</android.support.design.widget.CoordinatorLayout>

The code for the content_main.xml is given below:


<?xml version="1.0" encoding="utf-8"?>
<android.support.constraint.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:app="http://schemas.android.com/apk/res-auto"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    app:layout_behavior="@string/appbar_scrolling_view_behavior"
    tools:context=".MainActivity"
    tools:showIn="@layout/activity_main">

    <fragment
        android:id="@+id/sceneForm"
        android:name="com.google.ar.sceneform.ux.ArFragment"
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        app:layout_constraintBottom_toBottomOf="parent"
        app:layout_constraintEnd_toEndOf="parent"
        app:layout_constraintStart_toStartOf="parent"
        app:layout_constraintTop_toTopOf="parent" />

</android.support.constraint.ConstraintLayout>

We’ve set the fragment as ArFragment. The ArFragment checks if the phone is compatible with ARCore.

Also, it checks if the camera permission is granted. If it isn’t, it’ll ask for it automatically.

Also, it’ll ask you to download the ARCore by Google Application.

ARCore is currently supported in this list of devices.

The code for MainActivity.java class is given below:


package com.journaldev.androidarsceneform;

import android.graphics.Point;
import android.net.Uri;
import android.os.Bundle;
import android.support.annotation.Nullable;
import android.support.design.widget.FloatingActionButton;
import android.support.v7.app.AlertDialog;
import android.support.v7.app.AppCompatActivity;
import android.support.v7.widget.Toolbar;
import android.view.View;
import android.view.Menu;
import android.view.MenuItem;

import com.google.ar.core.Anchor;
import com.google.ar.core.Frame;
import com.google.ar.core.HitResult;
import com.google.ar.core.Plane;
import com.google.ar.core.Trackable;
import com.google.ar.sceneform.AnchorNode;
import com.google.ar.sceneform.rendering.ModelRenderable;
import com.google.ar.sceneform.ux.ArFragment;
import com.google.ar.sceneform.ux.TransformableNode;

import java.util.List;
import java.util.function.Consumer;
import java.util.function.Function;

public class MainActivity extends AppCompatActivity {

    ArFragment arFragment;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        Toolbar toolbar = (Toolbar) findViewById(R.id.toolbar);
        setSupportActionBar(toolbar);

        FloatingActionButton fab = (FloatingActionButton) findViewById(R.id.fab);
        fab.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View view) {
                addObject(Uri.parse("Coffee Cup_Final.sfb"));
            }
        });

        arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.sceneForm);
    }

    @Override
    public boolean onCreateOptionsMenu(Menu menu) {
        // Inflate the menu; this adds items to the action bar if it is present.
        getMenuInflater().inflate(R.menu.menu_main, menu);
        return true;
    }

    @Override
    public boolean onOptionsItemSelected(MenuItem item) {
        // Handle action bar item clicks here. The action bar will
        // automatically handle clicks on the Home/Up button, so long
        // as you specify a parent activity in AndroidManifest.xml.
        int id = item.getItemId();

        //noinspection SimplifiableIfStatement
        if (id == R.id.action_settings) {
            return true;
        }

        return super.onOptionsItemSelected(item);
    }

    private void addObject(Uri parse) {
        Frame frame = arFragment.getArSceneView().getArFrame();
        Point point = getScreenCenter();
        if (frame != null) {
            List<HitResult> hits = frame.hitTest((float) point.x, (float) point.y);

            for (int i = 0; i < hits.size(); i++) {
                Trackable trackable = hits.get(i).getTrackable();
                if (trackable instanceof Plane && ((Plane) trackable).isPoseInPolygon(hits.get(i).getHitPose())) {
                    placeObject(arFragment, hits.get(i).createAnchor(), parse);
                }
            }
        }
    }

    private final void placeObject(final ArFragment fragment, final Anchor createAnchor, Uri model) {
        ModelRenderable.builder().setSource(fragment.getContext(), model).build().thenAccept((new Consumer() {
            public void accept(Object var1) {
                this.accept((ModelRenderable) var1);
            }

            public final void accept(ModelRenderable it) {
                if (it != null)
                    MainActivity.this.addNode(arFragment, createAnchor, it);
            }
        })).exceptionally((new Function() {
            public Object apply(Object var1) {
                return this.apply((Throwable) var1);
            }

            @Nullable
            public final Void apply(Throwable it) {
                AlertDialog.Builder builder = new AlertDialog.Builder(MainActivity.this);
                builder.setMessage(it.getMessage()).setTitle("error!");
                AlertDialog dialog = builder.create();
                dialog.show();
                return null;
            }
        }));
    }

    private void addNode(ArFragment fragment, Anchor createAnchor, ModelRenderable renderable) {

        AnchorNode anchorNode = new AnchorNode(createAnchor);
        TransformableNode transformableNode = new TransformableNode(fragment.getTransformationSystem());
        transformableNode.setRenderable(renderable);
        transformableNode.setParent(anchorNode);
        fragment.getArSceneView().getScene().addChild(anchorNode);
        transformableNode.select();
    }

    private Point getScreenCenter() {
        View vw = findViewById(android.R.id.content);
        return new Point(vw.getWidth() / 2, vw.getHeight() / 2);
    }
}

The anchor is the position where the model or the Node is placed in the screen. It’s the center of the screen here.

In the addObject() the ARFragment gets the hit points on the plane using motion tracking.

placeObject() asynchronously sets the node to the center of the screen.

Following are the outputs when the application is launched:

android ARCore example

android augmented reality example app

augmented reality in android apps example

In the final output, the cup is positioned on the plane which consists of white dots.

The above screenshot is from Samsung S8.

Now try changing the model scale in SFA file!

This brings an end to this tutorial. You can download the final AndroidArSceneform Project from the link below.

The size of the project is big, thanks to the OBJ file.

You can also download the android studio project from our Github Repository.

Comments

  1. Joel says:

    Hello!
    Maybe you can still help me, if you see this at all.
    What I wanted to do was , instead of using the button ,I wish to something when I click the 3d model itself.
    When I try to do what you did to the button to a node I always get an error

    1. mdshahim says:

      I want to place an object without tapping on a particular location how to place ?

  2. dheeraj kumar says:

    hi,

    i have seen your tutorials for how to use AR core but your code is not working properly for detect image with obj file

  3. Lynn Myat Minn says:

    Hello Anupam Chugh,
    I am newbie to ARCore, Sceneform and I’m looking for some tutorial or sample of targeting image with buttons (maybe buttons are floating just like in Sceneform Solar System Sample). I hope you can help me! Thanks for your great tutorial sir.

  4. krutika says:

    Not working

    1. Anupam says:

      Can you elaborate on the error you’re getting?

  5. pradeep trivedi says:

    hello sir i am facing error Coffee Cup_Final.sfb file not found , how to solve please guide me

    1. captain says:

      i modified in assets the file to coffe.sfb, and it worked

Leave a Reply

Your email address will not be published. Required fields are marked *

close
Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages